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EXECUTIVE SUMMARY 

The Cooperative Demonstration Program (High Technology) was 
established by Congress to support school districts, community col- 
leges, consortia, and private industry in demonstrating new approaches 
to vocational education. In FY1988, the first year of the program, the 
Office of Vocational and Adult Education (OVAE), U.S. Department ov 
Education, awarded 36 grants totaling $9.5 million. The following 
fiscal year, OVAE awarded an additional 30 grants totaling $9.2 
million. Although all grantees proposed vocational education training 
for participants, the type of participants to be trained, the content 
of training, and the length of training varied widely across projects. 
Approximately half of the grantees also pr posed to develop new voca- 
tional education curricula. 

An analysis of the grant applications submitted by the grantees 
showed that only 12 of the 66 applications presented a clear and 
coherent design for their project, suggesting they would be very likely 
to be successful in their demonstration. An additional 26 presented a 
moderately clear and coherent project design, suggesting they might be 
somewhat less likely to be successful. 

All grantees were able to accomplish at least a portion of the 
activities proposed in their application. Projects tended to focus on 
short-term training, and the typical participant received fewer than 
100 hours of training. In projects with a large number of private 
partners, those partners tended to be simply customers of the project's 
customized training, while in projects with fewer partners, those 
partners tended to share in the design and/or delivery of training. 
All projects encompassed some aspect of high technology, even if just 
using high-tech equipment to teach low-tech job skills. Projects that 
were not able to accomplish all of their proposed activities were 
hampered by: their inability to recruit special populations, poorly 
developed project designs, failed partnerships, and overly ambitious 
objectives for the project (e.g., development of an interactive laser 
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disk)- The 18-month time frame for the grant proved too short for 
projects to accomplish much more than customized training* 

Although there is no OVAE standard for "reasonable" costs asso- 
ciated with demonstration projects, the per-participant per-hour costs 
of training tended to be similar across projects and, therefore, may be 
considered reasonable relative to one another* The cost for developing 
new curriculum materials varied widely, depending upon the technology 
used to present the information. 

A. Purpose of the Cooperative Demonstration Program 

The Cooperative Demonstration Program (PL 98-524, Title IV, Part 
B, Siibpart 1) was the largest demonstration effort supported under the 
Carl D. Perkins Vocational Education Act of 1984. The program provided 
the U.S. Department of Education (ED) and educational institutions an 
opportunity to try new approaches to vocational education and to learn 
about the effectiveness of these approaches. Funded projects were to 
reflect the Act's priorities: 

■ Increased access to high quality programs for 
special populations; and 

■ The overall improvement of the quality of 
vocational education. 

Projects also were to demonstrate successful cooperation among 
private employers and public agencies that resulted in training in 
advanced vocational education skills. A variety of models were 
suggested in the legislation, including work experience and apprentice- 
ship, worksite training, placement, and public works. Eligible appli- 
cants included State education agencies, local education agencies, 
postsecondary educational institutions^ institutions of higher educa- 
tion, and other public and private agencies, organizations, and insti- 
tutions. Of 181 applications submitted in FY1988 (the Programs's first 
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year), 36 were approved and awarded grants ranging in size from $50,000 
to $550,000. In FY1989, 136 applications were submit d, and 30 fiere 
awarded grants ranging in size from $171,874 to $473,549. 

All projects had to serve people enrolled in vocational programs 
directly and be widely replicable by service providers. Furthermore, 
grant recipients had to provide, through cash or in-kind contributions, 
a minimum of 25 percent of the demonstration project's total cost. 
Contributions could include the fair market value of facilities, 
overhead, personnel, and equipment. 

In addition to the priorities contained in the Act, each year the 
Secretary of Education had the prerogative to establish additional 
priorities for the program. In the first year of the program (FY1988), 
an invitational priority was issued by the Secretary for projects 
addressing high technology, but not all projects funded in FY1988 
responded to the invitation. Of the 36 grants awarded, OVAE identified 
23 as being "high technology," based on either the type of job for 
which training was conducted (or a curriculum developed) or the nature 
of the training given students. These 23 projects were the focus of 
the first-year evaluation effort. In FY1989, high-technology was made 
an absolute priority and all 30 projects funded in FY1989 addressed 
high-technology issues. 



B. Findings of the Evaluation 

This report presents the findings from an evaluation of high-tech- 
nology projects funded by the Cooperative Demonstration Program in 
FY1988 and FY1989. The evaluation answered three important questions: 

■ Did the grant applications present a clear 
and coherent design for a project? 

■ Were the grantees able to implement their 
projects as proposed, and, if not, what 
problems affected their implementation? 
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■ Were project costs reasonable in relation to 
projected or actual outcomes? 

The answers to these questions arc presented in separate sections in 
the report and summarized in the following paragraphs. 

Clear and Coherent Pro.iect Design 

Of the 23 FY1988 applications analyzed, only three presented a 
clear and coherent design for a project while an additional ten were 
moderately clear and coherent in their design. These applications were 
judged to be, respectively, highly likely and moderately likely to be 
successful. The remaining ten applications were much less clear in 
their design and, accordingly, judged much less likely to be success- 
ful. 

The quality of the awarded applications improved in the second 
year of the program. Of the 30 FY1989 applications reviewed, nine 
presented a clear and coherent design, while an additional 16 were 
moderately clear and coherent in their design. The remaining five 
applications were much less clear in their design and judged less 
likely to be successful. 

Project Implementation 

The evaluation focused on five aspects of project implementation: 
project training, public/private partnerships, the involvement with 
high technology, the operation of the project within the grantee insti- 
tution, and the repl icabil i ty or "exportabil ity" of the projects 
activities and products. 

Project Training . Overall, projects focused on providing rela- 
tively short-term training. The typical participant received fewer 
than 100 hours of training. The projects included in the site visits 
can be grouped into a few basic categories: 

■ Short-term, skill -specific instruction for 
current employees of companies, or for per- 
sons already familiar with the field of 



ERIC 



6 



vii 

training* This approach was most common 
among manufacturing projects, but it could 
also be seen in the refresher nursing course 
and in business projects teaching specific 
software applications, as well as home auto- 
mation installation* Sometimes instruction 
was customized, tailored to the needs of 
specific employers* 

■ Medium-term, entry-level technician training. 
These were programs in which training ranged 
from a few months to a year, often aimed at 
helping unemployed or otherwise disadvantaged 
persons find jobs in a field. These programs 
cut across the industries but were quite 
common in business. The three projects run 
by JTPA-supported agencies fell within this 
category, one of which provided training in 
several fields. Some provixied certificates* 

■ Longer-term programs aimed at two-year certi- 
fication or degrees (associate degree, li- 
cense)* Some of these projects were aimed at 
institutional capacity-building and regular 
students, e.g., creating new or revising 
existing offerings within departments of 
industrial technology or business/computers 
in community colleges. One was an appren- 
ticeship training project, and one an occu- 
pational program for high school students. 



Partnerships . Among projects included in site visits, intensities 
of partnerships varied from two projects where single employers played 
critical roles in service development and delivery, to several projects 
in which employers did little more than attend meetings of an advisory 
committee* According to survey results, most partnerships fit one of 
two models: 



■ The partner was a customer of the project, 
e.g. the project provided customized training 
to the partners' employees: or 

■ The partner shared actively in the delivery 
of instruction and services. 
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Projects with multiple partners were considerably more likely to report 
that partners were customers, while projects with few partners used 
partners in more ways, including active sharing of instruction. 

High Technology . Initial concerns that projects might not have a 
high-tech focus do not appear to be have been reflected in practice. 
This is true primarily because almost all fields have some high-tech 
elements. If learning to operate high-tech equipment is included as a 
training option, requiring that the project be high-tech in focus 
excludes almost no training effort. 

Operation and Integration of Projects within Institutions . FY1989 
projects reported fewer start up problems than did FY1988 projects. 
The IB-month time frame continued to present problems in completing 
project activities however. Whether a project started providing 
services shortly after award made little difference in whether it 
continued after the grant ended. The site visit teams identified 
several problems that cut across projects. They included: an inabil- 
ity to recruit special populations as planned, problems in getting and 
maintaining plant and materials, poorly developed designs that could 
not be executed, and underestimating the time necessary for product 
development. Aside from the recruitment difficulties, most problems 
were resolved but had an impact on those projects' ability to complete 
work within the 18-month grant time frame. The poor economy did not 
affect the majority of projects--especially projects in industries such 
as health care where the conditions may not have been as bad. 

Exportabilitv of Pro.iect Activities and Products . Survey results 
suggested that projects lacked consensus on what it was that the 
Cooperative Demonstration Program was "demonstrating." Only 20 
projects considered their public-partnerships, and only 13 projects 
considered their training, as applicable to, or providing a useful 
model, for others. Site visits revealed that most projects simply did 
not conduct the kinds of evaluations that would allow possible repli- 
cators to determine whether the project was successful for partici- 
pants. Projects that developed and tested discrete curricula or 
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products were more likely to have some evidence of effectiveness and a 
product to disseminate. 

Project Costs 

Ideally, the evaluation would determine cost effectiveness by 
matching costs to specific activities and outcomes. However, the study 
team was not able to collect cost data for each outcome separately for 
two reasons. First, grantees did not code expenditures by activity 
because their accounting system did not operate at that level of 
detail, i.e., expenditures were aggregated only according to budget 
line items. Second, several activities usually contributed to the same 
outcomes-projects implemented activities or one activity was used to 
support more than one objective. 

The team analyzed the three activities for which cost data were 
available through the grantee's accounting system or the final contract 
budget: planning and administration, student training, and curriculum 
development. Planning and administration costs include wages of the 
project director and clerical staff, associated fringe benefits, other 
direct costs associated with administration, and indirect costs. 
Student training costs include the wages of instructors and other 
specialists, associated fringe benefits, other direct costs (e.g., 
textbooks, supplies, travel, and stipends), and indirect costs. Cur- 
riculum development costs include the wages of instructors and cur- 
riculum development specialists, associated fringe benefits, other 
direct costs (e.g., training workshops, travel, and printing), and 
indirect costs. 

Treatment Costs . The first major analysis separated planning 
costs from the costs of proviaing the service. Treatment costs equal 
total costs less planning and administrative costs: 

Treatment Costs = (total project costs) - (project director 
wages + clerical wages + fringe benefits + 
other direct costs + related indirect costs) 
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The proportion of total project resources devoted to planning and 
administration ranged from 9.5 percent to 48.0 percent. The mean 
proportion of resources was 23.5 percent, and the median proportion was 
28.8 percent. 

Pro.iect Intensity . The intensity of the project (i.e., the number 
of successful outcomes relative to the effort expended to accomplish 
those outcomes) is defined as: 

Project Intensity = total treatment hours 

number of successful completions 

The effort to train students was the sum of the hours students spent in 
training in all courses. The number of successful outcomes was the sum 
of the students completing training. In projects where the treatment 
was staff training, the number of successful outcomes was the number of 
teachers completing training. Project intensity for each activity for 
which data were available--training students and training staff--ranged 
from 16.92 hours to 7,000 hours. The mean intensity of student 
training was 1,23C hours, and the median intensity was 3,508 hours. 
Staff training, offered formally at only three sites, ranged from 20 
hours to 53.5 hours per teacher. The mean number of hours per teacher 
was 38.8 and the median number of hours was 36.8. 

Average Unit Cost of Services . The average cost per unit of 
service is the total cost of the service divided by the number of units 
provided: 

Average Unit Cost = total treatment cost 
of Service total units of service provided 

For example, total service costs for providing student training was the 
sum of the project costs for staff, materials, overhead, etc. for all 
the courses. Total service costs for curriculum development was the 
sum of project costs for staff, testing, reproduction, overhead, etc., 
for all the courses produced. Total units of service provided was the 
sum of all students entering training or the number of new courses. 
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The average unit cost for student training ranged from $.45 to $40.47. 
The mean average unit cost for student training was $27.59, and the 
median average unit cost was $20.46. 

Comparing the average cost-per-hour-of-training across projects 
may create an unfair comparison because of variations in the intensity 
of the training and the number of students being trained. These dif- 
ferences affect the comparison of costs for curriculum development. 
The average unit cost for curriculum development (cost per course hour) 
ranged from $1.69 to $24,179.90. The mean average unit cost for cur- 
riculum development was $2,613.24, and the median average unit cost was 
$12,090.79. However, comparing the average cost-per-unit-hour for cur- 
riculum development also may be misleading. One project's course was 
an interactive videodisc, which had a much higher initial development 
cost due to the technology used. 

To provide a more accurate comparison, the analysis should com- 
pute the average cost per hour of training per student trained . 



The per-hour per-student costs ranqe from $.001 to $9.24. The mean 
per-hour per-student cost was $1.95, and the median per-hour per- 
student cost was $4.62. 

Service Cost Per Unit of Outcome . The service cost per unit of 
outcome is defined as total service costs divided by the number of 
completions: 

Service Cost « total service costs 

per Unit of Outcome number of successful completions. 

The number of successful completions is defined as the sum of the stu- 
dents completing the training and the number of courses successfully 
developed. The unit service costs for training students ranged from 
$190 to $62,971. The mean unit service cost for training students was 



Average Unit Cost 
of Service per 
Unit of Outcome 



total units of service provided 



total service costs 



number of completers 
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$8,695, and the median unit service cost was $31,580. For curriculum 
development, the cost per course ranged from $233 to $241,799. The 
relatively high cost at one project again was due to the high cost of 
the interactive videodisc. The mean unit service cost for curriculum 
development was $41,908, and the median unit service cost was $121,249. 

In summary, the answer to the question "are project costs reason- 
able in relation to project outcomes?" appears to be yes for all pro- 
jects. Neither OVAE nor the literature of vocational education have 
defined the absolute cost standards against which to measure "reason- 
ableness" of demonstration activities such as training or curriculum 
development. Thus, it is not possible to conclude that project costs 
were either reasonable or unreasonable in an absolute sense. However, 
the fact that the per-unit and per-outcome costs for the 19 indepen- 
dently-operated projects tended to cluster in the same area (even 
though total costs and project intensity varied substantially) suggests 
that project costs were reasonable relative to one another. 

L Lessons for Federal Program and Demonstration Management 

Although the Cooperative Demonstration Program (High Technology) 
has ended, the evaluation of the program has yielded important lessons 
that can be applied to future demonstrations sponsored by OVAE and the 
Department. The information obtained from the 27 FY1988 and FY1989 
grantees visited by the study team and the 30 FY1989 grantees surveyed 
suggest ten lessons regarding project design, project implementation, 
and project costs. These lessons are presented in the following para- 
graphs, organized according to the three evaluation topics: project 
design, implementation, and costs. 
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Lessons Regarding Project Design and Presentation in the Grant 
Appi i cation . 

Lesson 1: Program regulations should be more narrowly 
defined to convey explicit information about the kinds of 
interventions sought . 

Although titled "Cooperative Demonstration Programs," the autho- 
rizing legislation did not narrowly define the nature of the demon- 
stration. It was not clear whether the funded projects were to demon- 
strate that a particular program can be successful if it has not been 
tried before, can be improved in its original site if already opera- 
ting, can be successful in a new site if already implemented elsewhere, 
or some combination of these intentions. The broad definition of 
acceptable interventions allowed a wide variety of projects to be 
funded. Although all projects satisfied the general conditions of the 
program's regulations, it is uncertain whether all projects met the 
intent of the regulations and the Perkins Act. For example, was a 
grant to a single company to retrain its current employees a legitimate 
purpose under the program? OVAE should state whether the demonstration 
is to field test previously identified public/private partnerships or 
to develop new models of replicable partnerships. Similarly, was a 
project using high technology methods (e.g., interactive video or 
computer assisted instruction) to train students for low-tech jobs a 
legitimate use of Federal funds? If operating high-tech equipment is 
included as a training option, requiring that the project be high-tech 
allows almost any training effort to be included. 

Lesson 2: Applicants should be required to show the logic 
and plausibility of their project designs . 

In general, demonstration projects are likely to be most success- 
ful if they propose a logical design of activities to be accomplished 
and objectives to be achieved. A logical project design identified the 
linkages between resources and activities, activities and short-term 
outcomes, and short-term outcomes and long-term outcomes. Applicants 
for Cooperative Demonstration grants often were unclear in their logic 
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regarding how resources would be used to conduct the activities. For 
example, many projects indicated that Federal funds would be used to 
hire staff, but they often failed to report precisely what the staff 
would be doing. Future applicants should be required to explicitly 
show the logic of the project design. 

A plausible project design has some likelihood of achieving its 
short-term objectives, where short-term is defined as the grant award 
period. Project objectives may be implausible because: 1) schedules 
are unrealistic; 2) resources are insufficient; or 3) available know- 
ledge suggests that the project is not likely to achieve its objectives 
(e.g., objectives may be too ambitious or the staff may not posses the 
requisite skills). As with logic, applicants should be required to be 
explicit regarding the plausibility of the project design. 

The panel reviewing the applications should include experts who 
are familiar with the high technology field or activities being pro- 
posed; and who can judge plausibility. Such experts could identify 
activities (e.g., the development of interactive videodiscs), which are 
not likely to be feasible within the time frame or resources proposed. 
The experts also could assess whether the applicant's experience in a 
field is sufficient to successfully accomplish more complicated activi- 
ties, e.g., the construction of a Class I clean room for manufacturing 
computer chips. 

For a project to have a good chance of succeeding, it should rank 
high on both logic and plausibility. Two high rankings suggest the 
project is likely to accomplish the activities and objectives stated in 
the application. By contrast, projects receiving a low ranking in 
either logic or plausibility begin with the disadvantage of little or 
poor planning. Furthermore, if logic and plausibility are important to 
the success of the project, OVAE should award points for them in the 
evaluation criteria. Applicants take their cue from the point award 
criteria in the grant application package to emphasize certain areas 
over others. No points were given for logic and plausibility, even 
though these have a serious impact on potential success. 
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Lessons Regarding Project Implementation . 

Lesson 3: Grant awards should be made in the spring for pro- 
.iect starts in the summer because of the greater availability 
of grantee staff, reduced training demands bv other programs, 
and more opportunities for planning . 

The timing of the award notification and starting date have impor- 
tant implications for project implementation. Most academic institu- 
tions plan their staffing and activities according to the academic-year 
calendar. Academic institutions plan in the spring for the following 
year and try to have both activity schedule and staff assignments 
settled by May or June preceding the September start. 

Grants that start at the beginning of the academic year seem to be 
easier to implement than those starting mid-year. The difficulty with 
a mid-academic year start is that all available staff are already 
assigned to other projects, are in the midst of those assignments, and 
are reluctant to shift to a new assignment. If the grantee goes out- 
side the institution to hire project staff, there may be fewer quali- 
fied staff available mid-year. 

The major exception to scheduling project start-up for the begin- 
ning of the academic year is curriculum development projects that use 
significant amounts of instructor time to develop courses. In this 
type of project, the start of the curriculum development phase should 
coincide with the beginning of summer, when schools are out or teaching 
loads are light. In the summer, instructors have more time, are better 
able to concentrate, and will be able to integrate their new curriculum 
with their lesson plans for the fall semester. 

Lesson 4: The applicant's proposed training should be appro- 
priate to the labor needs of the geographic area surrounding 
the institution based on current labor market data . 

The labor market data used by most applicants described potential 

job openings and economic conditions for the state or region, rather 

than for the county or city in which the institution was located. 

These labor market data often were one to three years old at the time 

of the application. Although the data may have demonstrated a need for 
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job training, the jobs may have been located farther away than students 
were willing to coiTtmute (or relocate). The inability or unwillingness 
of students to take the jobs farther away negated the benefits of the 
training. Similarly, the jobs may have been available in the year(s) 
prior to the current recession or at the time a major employer was 
relocating to the area. These favorable conditions had changed by the 
time the students completed training (approximately 24 months after the 
application was submitted), and many students were unable to find 
employment in their area of training. Applicants should be required to 
justify the need for the grant based on recent data for the geogr?phic 
area in which students are receiving training. 

Lesson 5: Applicants should state in detail the role and 
responsibilities of the private partner(s) and include a 
statement from the partner agreeing to accept this role . 

The specific role of the private sector was not well defined in 

the program regulations and the private sector role varied widely 

across projects. Among the FY1989 projects, the private sector 

functioned as a: 

■ Grantee; 

■ Referral source for students; 

■ Employer of program graduates; 

■ Source of internships; 

■ Member of the advisory group; 

■ Supplier of equipment; or 

■ Source of instructors. 

Projects with multiple partners were considerably more likely to report 
that partners were customers, while projects with few partners used 
partners in more ways, including active sharing of instruction. 

The regulations and application requirements provided little 
guidance regarding private-sector involvement. Because the regulations 
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were vague about acceptable private-sector involvement, applicants 
similarly were vague in their descriptions of how private companies 
would be involved in their projects. The vagueness in the applications 
sometimes reflected incomplete planning on the part of the grantee. 
Other times it was the result of the grantee not being able to get the 
private partners to commit to training students or to activities other 
than advising the project. 

The applicant should be required to submit a letter from each 
partner stating that it is willing to commit to the level and type of 
support stated in the application. After the award, and during the 
planning stage, the grantee should establish a formal agreement with 
the partner detailing precise responsibilities. 

Lesson 6: Grantees should be encouraged to request changes 
to the scope of work in response to changes in the local 
labor market or other conditions . 

Several projects encountered operational problems during various 
stages of the grant, including a deteriorating or shifting labor 
market, withdrawal of a key partner, or discovery that an activity was 
more complicated or expensive than originally estimated. In reviewing 
the implementation history of the projects, the study team thought some 
problems were severe enough to warrant a change in project design or 
objectives, although few such changes were requested. For example, 
grantees initially targeting hard-to-serve populations tended to shift 
their focus to less challenging students once they realized the diffi- 
culty of what they proposed. Other projects were unrealistic in their 
expectations of clients' initial academic skills or the number of 
workers needed by employers. 

During the site visits, the teams asked project directors and 
staff why they did not request changes in grant scope to keep up with 
changing conditions. Almost all the project directors said that they 
thought they had to fulfill the terms of the application and that any 
requests for changes would be considered a sign of failure and/or not 
approved. This perception on the part of grantees resulted in projects 
continuing on an inappropriate--and sometimes unsuccessful --path. 
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Department program managers should discuss potential implementa- 
tion problems with grantees at the start of the grant. The discussion 
should include both the nature of the problems encountered by other 
demonstrations (not just the Cooperative Demonstration Program) and the 
conditions under which it is important and acceptable to request 
changes to the scope of the grant. Second, program managers should use 
the monitoring telephone calls to discuss both the accomplishments of 
the project and the problems (potential or actual) preventing expected 
accomplishments. If project directors are open about the problems they 
encounter, then the program manager will be better able to suggest 
solutions or put the project director in touch with other grantees 
having similar problems. This more open communication, however, will 
require that program managers become more active in the internal opera- 
tions of the project. 

Lesson 7: The length of the grant should allow sufficient 
time for pro.iect sta/t-up and accomplishment of the stated 
objectives . 

The duration of the grant has implications both for the activities 
that can be carried out and the ability to draw conclusions based on 
grant outcomes. The time period for the Cooperative Demonstration 
grants was 18 months, with some projects requesting three- or six-month 
extensions. For grantees who began new projects or had no existing 
staff to assign to the project, between three znd six months of the 
grant period was devoted to hiring staff, establishing office and 
contracting procedures, and planning training. For grantees with 
existing staff or a similar training activities already in place, 
startup time was not as great, and the training offered was often more 
extensive (e.g., part of an ongoing two-year training program). Almost 
all grantees agreed that 18 months was not enough time to complete 
training other than customized training. 

A more workable arrangement would disaggregate the award into two 
separate but related grant periods totaling 24 months. The first 
period would be a six-month planning award during which the grantee 
could update the labor market information on which the training is 



xix 

justified, conduct curriculuin development activities, and complete 
other startup activities. If labor market conditions or technology 
have changed, the training activities can be shifted accordingly. The 
planning award also would allow the grantee time to hire and orient the 
staff necessary to run the project. Finally, the planning grant allows 
time for the grantee to negotiate specific roles and responsibilities 
with the private partner. If the proposed priv:ite partner is unable to 
fulfill commitments made during the application process, the grantee 
has time to find an alternative partner(s). 

The second award would fund actual training or other proposed 
activities for 18 months. Because grantees find it difficult to 
attract project staff if there is no certainty of long-term employment, 
the operating award would be denied only if their plans were not 
adequately developed. The operating plan would be submitted as the 
final report of the planning grant. The OVAE program managers (and 
perhaps the original review panel) would be required to review and 
approve the operating plan before the second award could be released. 

Lesson 8: Grantees should limit the number of ob.iectives 
and be held more accountable for meeting the objectives 
stated in their applications or revised scopes of work . 

Applicants tended to state numerous objectives phrased in ways 

that made it difficult to measure success. Often the objectives 

referred to conditions that required more time than the grant period 

(e.g., produced journeyman technicians), required changes beyond the 

scope of the grant (e.g., make local employers more competitive), or 

were not time specific (e.g., meet employer's demand for trained 

workers) . 

Objectives should be stated in operational terms that can be 
measured by third-party evaluators, and the evaluations should focus on 
how well projects met those objectives. Although all projects had 
third- party evaluators, they were seldom used to measure the effective- 
ness of project activities or to determine the relative value of pro- 
ject components in the project's success. Instead, grantees used the 
outside evaluators to confirm that certain activities had taken place. 
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The evaluator would observe several classes, talk to several students 
and faculty, and produce a report of a few pages. In a few instances, 
evaluator was used to evaluate the curriculum materials used by the 
project rather than evaluate the project itself. 

Lessons Regarding Prelect Costs 

Lesson 9: Applicants should better substantiate the value 
assigned to in-kind contributions . 

The non-Federal contribution by grantees was composed primarily of 
1) grantee staff time and classroom space and 2) partner staff time and 
equipment donations. While the grantee staff time and space could be 
easily documented, the value assigned to partner staff and equipment 
contributions was simply self-declared without an independent 
appraisal. It is in the interest of the grantee to declare as high a 
value as possible for in-kind contributions so that they might more 
easily meet the minimum 25 percent contribution requirement as well as 
cast their application in a more favorable light by showing strong 
support from private industry. However, the possible inflated value of 
the donated equipment or partner staff time, also inflates the overall 
project cost data used in the cost benefit analysis of project out- 
comes. If the self-declared in-kind contribution is higher than the 
true fair market value, the costs per student trained, per staff member 
trained, per course developed, etc, would be incorrect and result in 
unreliable conclusions about the success of the demonstration. 

In reporting the self-declared value of the donated equipment, 
grantees failed to include in their proposed budget, the cost of trans- 
portation, installation, maintenance, and supplies for the equipment. 
These additional, unplanned costs sometimes required the reallocation 
of grant funds after the project started or prevented the grantee from 
utilizing the equipment as proposed. Encouraging applicants to recog- 
nize all the costs associated with accepting donated equipment should 
help prevent a shortage of resources during implementation. 
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Lesson 10: Grantees should be encouraged to build upon the 
existing literature, curricula, and other educational 
resources available in the field . 

Many of the projects engaged in curricula development either as a 
primary objective of the grant or in response to the requests for 
specific training from a partner organization. Rather than searching 
for existing course materials through ERIC or the OVAE curriculum 
centers and adapting these materials to the needs of the project, many 
projects developed their own curricula from scratch. Project directors 
reported they did not use--or at least review--the curriculum materials 
available elsewhere because 1) they considered their training to be 
"unique" and that materials developed elsewhere would not be relevant 
or 2) they were unaware that applicable materials were available else- 
where. A few project directors reported they checked with other 
grantees, but none reported conducting a systematic search of ERIC, the 
curriculum centers, or other clearinghouses. 

To avoid wasting resources on the development of curricula that 
have already been developed elsewhere (unless that is the specific 
purpose of the demonstration), OVAE should require applicants to show 
evidence of a systematic search for their curricula prior to the start 
of any new development efforts. If the Department thinks such a 
systematic search is too burdensome during the application stage, it 
should be required as part of the initial six-month planning grant. 
Even if a search is conducted during application preparation, an 
updated search should be made during the planning grant stage to ensure 
that no new curriculum packages, published during the previous 12 
months, are overlooked. 
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I. INTRODUCTION 

This report summarizes the activities of and findings from an 
evaluation of projects funded by the Cooperative Demonstration Program 
(High Technology) in FY1988 and FY1989. The evaluation answers three 
questions: Did grant applications present a clear and coherent design 
for a project? Were grantees able to implement their projects as 
proposed, and, if not, what problems prevented implementation? Were 
project costs reasonable in relation to projected or actual outcomes? 

Overview of the Cooperative Demonstration Program 

The Cooperative Demonstration Program (PL 98-524, Title IV, Part 
B, Subpart 1) was the largest demonstration effort supported under the 
Carl D. Perkins Vocational Education Act of 1984 (see Appendix A for a 
copy of the legislation). The program provided the U.S. Department of 
Education (ED) and educational institutions an opportunity to try new 
approaches to vocational education and to learn about the effectiveness 
of these approaches. Funded projects reflected the act's priorities: 
increased access to high quality programs for special populations and 
overall improvement of the quality of vocational education. Projects 
also were to demonstrate successful cooperation among private employers 
and public agencies that resulted in training in advanced vocational 
education skills. The regulations suggested a variety of models, 
including: work experience and apprenticeship, worksite training, 
placement, and public works. Agencies eligible to apply included State 
Education Agencies (SEAs), Local Education Agencies (LEAs), 
postsecondary educational institutions, institutions of higher 
education, and other public and private agencies, organizations, and 
institutions . 

Projects were authorized to be funded through grants, cooperative 
agreements, or contracts and could be: 

1. Model projects providing improved access to 
quality vocational education programs for-- 
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-Handicapped individuals; 
-Disadvantaged individuals; 
-Adults who are in nee:' of training 
and retraining; 

-Individuals who are single parents 
or homemakers; 

-Individuals who participate in programs 
designed to eliminate sex bias and 
stereotyping in vocational education; 

-Criminal offenders who are serving 
in a correctional institution; and 

-Men and women seeking nontraditional 
occupations. 



Projects that were examples of successful 
cooperation between the private sector 
(including employers, consortia of 
employers, labor organizations, and 
building trade councils) and public 
agencies in vocational education, including 
State boards and eligible recipients. The 
projects had to be designed to demonstrate 
ways in which vocational education and the 
private sector of the economy could work 
together effectively to assist vocational 
education students to attain the advanced 
level of skills needed to make the 
transition from school to productive 
employment, including 

-Work experience and apprenticeship 
projects; transitional worksite job 
training for vocational education students 
which is related to their occupational 
goals and closely linked to classroom and 
laboratory instruction provided by an 
el igible recipient; 

-Placement services in occupations which 
the students are preparing to enter; and 

-Where practical, projects that will 
benefit the public, such as the rehabili- 
tation of public schools or housing in 
inner cities or economically depressed 
rural areas. 

The projects may include institutional and 
on-the-job training, support services 
authorized by the Act, and such other 
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necessary assistance as the Secretary 
determines to be necessary for the 
successful completion of the project. 



3. Projects to overcome national skill 

shortages, as designated by the Secretary 
in cooperation with the Secretary of Labor, 
Secretary of Defense, and Secretary of 
Commerce. 



4. Such other activities which the Secretary 
could designate which are related to the 
purposes of the act. [ Federal Register . 
1985, pp. 33250-33251.] 



All projects, however, had to directly serve people enrolled in 
vocational programs and be widely replicable by service providers. 
Furthermore, grant recipients had to provide, through cash or in-kind 
contributions, a minimum of 25 percent of the demonstration project's 
total cost. Contributions could include the fair market value of 
facilities, overhead, personnel, and equipment. 

In addition to the authorized activities contained in the act, the 
Secretary of Education had the prerogative each year to establish 
priorities for the program. In the first year of the program (FY1988), 
an invitational priority (in addition to the absolute priority for 
authorized activities) was issued for projects addressing high 
technology. In FY1989, this invitational priority became another 
absolute priority. The term "high technology" was defined by the 
Perkins Act to mean: 



...state-of-the-art computer, microelectronic, 
hydraulic, pneumatic, laser, nuclear, chemical, 
telecommunication, and other technologies being 
used to enhance productivity in manufacturing, 
communication, transportation, agriculture, mining, 
energy, commercial, and similar economic activity, 
and to improve the provision of health care. [34 
CFR 400.4(b)] 



1-4 



The Application Process 

Projects funded by the program were proposed and implemented by 
educational institutions, private agencies, and other organizations. 
Each year since the program's inception in FY1988, the Office of 
Vocational and Adult Education (OVAE), U.S. Department of Education, 
received grant applications from agencies, institutions, and 
organizations interested in conducting demonstrations. Program 
guidelines were published in the Federal Register . In FY1988, 
application materials were mailed directly to prospective applicants; 
in FY1989, OVAE staff instructed applicants to photocopy the 
application and instructions contained in the Federal Register . 

In general, the application process was as follows. Applicants 
prepared and submitted project applications according to the published 
guidelines. Panels of outside readers, selected by OVAE, reviewed the 
applications and asked applicants for any necessary clarification. The 
applications were judged according to the following selection criteria 
and point allocations: 

■ Statement of need (15 points); 

■ Plan of operation (30 points); 

■ Quality of key personnel (10 points); 

■ Budget and cost effectiveness (10 points); 

■ Evaluation plan (5 points); 

■ Adequacy of resources (5 points); 

■ Private-sector involvement (10 points); 

■ Employment opportunities (5 points); and 

■ Dissemination (10 points). 

The projects receiving the highest scores were awarded grants, and as 
many projects were funded as the program budget would allow. In 
FY1988, a total of $9.5 million was awarded, and in FY1989 grant awards 
totaled $9.2 mill ion. 

Of 181 applications submitted in the programs' s first year 
(FY1988), 36 were approved and given grants ranging in size from 
$50,000 to $550,000. The winning projects were notified in October 
1988 and most began their work around January 1989. Although operating 
in calendar years 1989 and 1990, the first cohort of projects are 
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considered FY1988 projects because funding was appropriated from FY1988 
Perkins Act funds. 

In FY1989, 105 applications were submitted, and 30 received 
grants. Most of these projects began in January 1990, although at 
least two grantees who had received grants under the FY1988 cowipetition 
delayed starting until July 1990 in order to complete their FY1988 
projects. 

Not all FY1988 projects had responded to the Secretary's 
invitational priority of addressing high-technology issues. OVAE 
identified only 23 of the 35 funded projects as being "high 
technology," based either on the type of job for which training was 
conducted (or curriculum developed) or on the nature of the training 
given students. Those 23 projects were the focus of the first-year 
evaluation effort. All 30 projects funded in FY1989 addressed high- 
technology issues and were the focus of the second year evaluation 
effort. The lists of the FY1988 and FY1989 high technology projects 
are presented in Appendix B-1 and Appendix B-2. 

Defining Pro.iect Boundaries 

For this evaluation, "project" was defined as the activities 
funded by the Cooperative Demonstration grant and matching funds and 
occurring within the 18-month grant period. It did not include 
services provided before or after the grant period and benefits 
accruing after the end of the grant. Also excluded were the training 
activities and support services offered to project enrollees through 
other parts of the grantee institution. For example, a participant 
might enroll in training developed with project funds and provided by a 
project-supported instructor, using equipment donated by a local 
business as part of the non-Federal "match." Yet, to complete the 
degree or certificate for which the training was developed, the student 
might be expected (by the project design) to enroll in additional 
courses in the same institution that were not supported by the project. 
Students might also use financial aid or support services provided by 
or through the institution that were not part of the project. While 
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these activities might be necessary to complete the degree or 
certificate, only those directly funded by the grant were included in 
this evaluation. 

Boundaries of the project include interorganizational networks 
developed in the course of carrying out a project. Grantee 
institutions entered into a variety of formal and informal relation- 
ships with other organizations to offer students support services, 
provide them with jobs, or provide the project with additional 
financial or other assistance. 

Overview of this Report 

This final report is presented in six sections. Section I 
introduced the Cooperative Demonstration Program and the three major 
questions in the evaluation. Section II describes the modified 
evaluability assessment done for the 30 FY1989 projects and answers the 
first study question: were projects clearly and coherently designed? 
Section III describes the implementation of a subset of FY1989 
projects, based on a mail survey of 30 projects (27 projects responded) 
and site visits to 19 projects. Section IV analyzes project costs 
relative to project accomplishments and answers the third study 
question: were project costs "reasonable?" Section V examines two 
groups of issues related to the iianagement of the Federal program 
raised by the evaluation and suggests ways to improve future 
demonstration programs. Finally section VI summarizes the findings of 
the first four sections and recommends possible improvements in 
programs. The nine appendices describe the projects evaluated, the 
results of the mail survey, and the survey and field instruments. 
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II. AN ASSESSMENT OF PROJECT LOGIC AND DESIGN 



This section ranks the clarity and coherence of 30 grant applica- 
tions funded under the Cooperative Demonstration Program (High Tech- 
nology) in FY1989. Each grantee application was assigned a score 
according to its internal logic and the plausibility of its design. 
Originally, this evaluation had intended to relate the applications' 
scores to the ultimate success of the projects. Of the 30 grant appli- 
cations reviewed for logic and plausibility, four projects were 
predicted to be highly successful, 21 projects were predicted to be 
potentially successful, and five projects were predicted to be less 
successful. Ideally, only applications that offered well-defined sets 
of services and the possibility of success should be funded. 

Predicting the likelihood for success of the proposed projects can 
be accomplished, in part, using a methodology known as an evaluability 
assessment (EA). While EAs are traditionally used to determine the 
most appropriate design for a program evaluation, parts of the metho- 
dology can be useful for determining the logic or clarity and the 
plausibility or coherence of individual project designs. Part A 
describes the procedures used to develop a logic model for each funded 
project and to characterize the logic of the proposed design. Part B 
describes the procedures used to score the plausibility of each project 
design. Part C ranks the projects on both logic and plausibility to 
arrive at an overall prediction of success for the project. 
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A, Assessing the Logic of the Project Design 



The first part of the assessment created a logic model for each 
proposed project. Logical designs include well-defined project 
objectives and activities that are linked to those objectives. 
Projects are more likely to succeed if they propose a clear and logical 
design of objectives to be achieved and activities to be accomplished. 
Designs can be detailed but illogical if activities do not proceed 
systematically from objectives. Unclear designs, however, cannot be 
logical because there is insufficient information to trace the logic 
linking the components. Projects may be uneven, with some clear and 
logical components and others that are not. 

To determine each application's logic, the study team reviewed 
each grant application to identify the following: 

■ An explicitly stated set of inputs or 
resources; 

■ An explicitly stated set of activities or 
events; 

■ An explicitly stated set of outcomes (short- 
term and long-term); and 

■ An explicitly stated set of causal links 
among events and outcomes that establish the 
flow of effects expected from the project. 

The study team recorded the components of each of the 30 grant 
applications in a logic shell illustrating the linkages. Two 
contrasting examples of logic models illustrating the procedure are 
provided. Figure II-l displays a logic model where several events are 
linked with project outcomes, demonstrating a clear and logical design. 
In contrast, Figure II-2 displays a project design where no such 
linkages are evident. Scoring each design reflects whether the project 
posses a logical design according to procedures described below. 
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Developing the Logic Model 

The study team followed a six-step process when reviewing grant 
applications provided by the OVAE program office. The team assembled 
all material pertaining to the grant application and pre -sal, 
including: the original proposal; any revisions to the original 
proposal; correspondence dated prior to the award date of the grant; 
supplementary material submitted by the applicant prior to the date of 
award; and grant award documents. 

Next, the team identified long-term outcomes from information 
contained in each application's introduction, statement of need, and 
objectives. Long-term outcomes were entered in the logic model shell. 
The team recorded all information as stated in the application and, 
when information was available, specified who the project was intended 
to serve. The team next entered short-term objectives from information 
in the "plan of operation" section of the application. Then the team 
entered the resources available to the project on the first day of the 
grant award from the budget, introduction, and other parts of the 
appl ication. 

Finally, the team showed the stated causal links between inputs, 
activities, short-term goals, and long-term goals with directional 
arrows, based on reviewing budget narrative for explanations of how 
Federal and non-Federal funds were to be spent. A directional arrow 
was drawn when the narrative matched activities specified in the 
diagram. The team reviewed the remainder of the application for other 
logical relationships stated explicitly within the same paragraph in 
the proposal. Because paragraphs should represent complete thoughts, 
linkages were designated only if two elements were expressed within the 
same parayraph. This is a restrictive approach and affects the 
projects' final rankings. Without such restrictions, however, the team 
could not determine objectively if or how the applicants intended to 
connect the activities. Boxes without connecting arrows represent 
logical gaps. 
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Scoring Project Logic 

Procedure. Applications were scored in a three-step process. 
First, the study team counted the number of inputs, activities, short- 
term outcomes, and long-term outcomes. Totals for each are shown in 
the summary of "Raw Scores of Project Logic," presented in Appendix 
C-1. 

Then the team determined what activities were identified for 
arf-jeving each short-term outcome. The team assigned one point for 
each short-term outcome supported by at least one activity and 
subtracted one point for each short-term outcome not so supported. 

Next, the team determined the short-term outcomes identified for 
achieving tich long-term outcome. Again, the team assigned one point 
for each long-term outcome supported by at least one short-term outcome 
an'l subtracted one point for each long-term outcome not so supported. 

Analysis . The design of the project was considered clear and 
lo{jica1 if it met five conditions. First, the design must have 
identified at least one long-term outcome, e.g., access to quality 
vocational education training or enhancement of the competitive 
strength of local industry. Second, the design had to identify one or 
more short-term outcomes to be achieved during the grant period that 
would help achieve long-term outcomes. Third, the design had to 
specify one or more activities during the grant period that would help 
achieve a short-term or long-term outcome. Fourth, the design had to 
have had more linkages than gaps between activities and short-term 
outcomes. Total points assigned to linkages or gaps, therefore, must 
yield a positive number to be considered logical, e.g., a project that 
had four explicit linkages and two gaps had a total score on this 
criterion of +2. Fifth, the design had to have more linkages than gaps 
between short-term outcomes and long-term outcomes. The total points 
assigned to linkages or gaps had to be a positive- number, e.g., a 
project that had four stated linkages and three gaps had a total score 
on this criterion of +1. Raw scores for the projects are shown in 
Appendix C-1. 
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Results of the Logic Assessment 

Description of Pro.iect Logic . Using these procedures, the study 
team assigned each project an overall score for logic. After testing 
for the presence of all five conditions, it found: 

■ All 30 projects identified one or more long- 
term objectives; 

■ Eleven projects linked one or more short-term 
outcomes to a long-term outcome; 

■ Twenty-nine projects linked one or more 
activities to a short-term outcome; 

■ Twenty-three projects displayed more linkages 
than gaps between activities and short-term 
outcomes; and 

■ Only two projects displayed more linkages 
than gaps between short--orm and long-term 
outcomes. 

Table II-l ranks each project according to the five conditions and 
shows total scores. Based on the overall scores, nine projects 
received a high rating; 16 received a medium rating; and five received 
a low rating. Table II-2 summarizes the logic rankings of all the 
projects. 

Most applications met two criteria: they identified lohg-term 
outcomes and linked proposed activities to short-term outcomes. To 
demonstrate a logical design, however, the proposals also needed to 
identify linkages between short-term and long-term outcomes. Only two 
projects had more linkages than gaps between short-term and long-term 
outcomes. Further, application's often were unclear about how 
resources would be used to conduct the activities. Many projects 
indicated that Federal funds would be used to hire staff. They often 
failed, however, to report precisely what staff would be doing. In 
summary, most applications did not explicitly link resources with 
activities. 
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B, Assessing the Plausibility of Project Design 



Defining Plausibil itv 

A plausible project is one with some likelihood of achieving its 
short-term objectives (short-term is defined as the 18 months of the 
grant award period). Project objectives may be implausible because 
schedules are unrealistic, resources are insufficient, or available 
knowledge suggests that the projects are unlikely to achieve objectives 
(e.g., too ambitious, or unskilled staff). Thus, assessing the 
plausibility of the project's design means determining the degree to 
which a project: 

■ Is well defined; 

■ Describes feasible relationships among 
components; 

■ Can be completed with available resources; 
and 

» Can be completed within the specified time 
period. 

For each major short-term outcome, the study team asked the 
following questions: Are resources adequate to achieve outcomes? Is 
the schedule for achieving the outcome reasonable? Do the activities 
suggest an understanding of the steps necessary to achieve the outcome? 
The answers to these questions were recorded in Table II-3. 

Procedures for Assessing Plausibility 

The team followed a five-step process to determine project 
plausibility. First, it reviewed the logic models already developed as 
well as the grant application to determine whether resources were 
adequate for each activity. One point was assigned to each activity 
supported by at least one resource, and one point was subtracted for 
each activity not so supported. Point totals are shown in the summary 
of "Raw Scores of Project Plausibility," in Appendix C-2. 
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Second, the study team assessed whether all proposed activities 
could be completed within the 18-month grant period. Activities judged 
doable were awarded one point; one point was subtracted for each 
activity that could not be completed in 18 months. 

Third, the study team determined whether all the proposed short- 
term outcomes could be achieved within the 18-month grant period. One 
point was assigned to those that could be achieved within the grant 
period; one point was subtracted for each that could not be achieved. 

Fourth, the study team determined whether the overall project 
design and outcomes (long-term and short-term) adequately 
reflected the two objectives required for the Cooperative Demonstration 
Program: 

1. Access to quality vocational education 
training; and 

2. Successful cooperation between public and 
private sectors. 

One point was assigned if the project design and/or long-term or short- 
term outcomes adequately reflected the required objectives for the 
Cooperative Demonstration Program. 

The design of the project was considered highly plausible if it 
met all four conditions. Projects that met three conditions were 
considered "plausible." Projects that met fewer than three criteria 
were considered "less plausible". 

Results of the Plausibility Assessment 

Description of the Plausibility Assessment . Thirty FY1989 
projects were scored for project plausibility, and the results are 
shown in Table II-3. It was found that: 

■ Twenty-three of the 30 projects appeared to 
allocate sufficient resources to conduct 
their activities; 
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■ Twenty-nine projects appeared capable of 
completing the proposed activities within 18- 
months; 

■ All thirty projects appeared capable of 
achieving the proposed short-term outcomes in 
the 18-month time frame; and 

■ Twenty-one projects reflected the two 
objectives of the Cooperative Demonstration 
Program: access to training and a successful 
public/private partnership. 



Analysis of the Plausibility Assessment . Thirteen projects were 
scored as highly plausible, and 17 projects were scored as plausible 
(see Table II-4). For an example, the application submitted by Alabama 
Aviation College met all four conditions. This project developed a 
training model and curriculum in aviation maintenance targeted toward 
minorities and females (underrepresented groups). Outcomes specifi- 
cally reflected the objectives of the program, i.e., access for under- 
represented groups and a successful public/private partnership. 
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C. Summary of the Evaluabilitv Assessment 



Thirty FY1989 projects were assessed on the logic and plausibility 
of their project design as stated in their grant applications. On 
logic, nine were rated high, 16 medium, and five low. On plausibility, 
13 were highly plausible, and 17 were plausible — no projects were "less 
plausible. " 

A project with a good chance of succeeding should rank high on 
both logic and plausibility. By contrast, projects receiving a low 
rating in either logic or plausibility began with the disadvantage of 
little or poor planning. Table II-5 shows the logic and plausibility 
rankings of all 30 projects. Accounting for both logic and plausibi- 
lity, four projects were predicted to be highly successful, 21 projects 
were moderately successful, and five projects less successful. Table 
II-6 summarizes the predictions for project success. 

Comparing FY1988 and FY1989 Projects 

Table II-7 compares final rankings in project logic for FY1988 and 
FY1989 applications— showing similar distribution of applications 
across categories. FY1988 and FY1989 applications differed on scores 
for project plausibility (Table II-8). More FY1989 than FY1988 
applications scored high in project plausibility. No FY1989 projects 
scored low in project plausibility, while seven of the FY1988 projects 
received low plausibility scores. 

In rankings of logic and plausibility used to predict project 
success, FY1988 and FY1989 projects showed similar rates for high 
success (Table II-9). However, FY1989 projects were predicted to be 
potentially successful at twice the rate of FY1988 projects. This was 
due, in part, to the fact that reviewers did not require FY1989 
projects to mention explicitly a successful public/private partnership 
as one of its outcomes. If projects reflected the public/private 
partnership in its overall design and/or outcomes, grantees were 
awarded one point. 
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Table II-5 

SUMMARY OF PROJECT LOGIC AND PLAUSIBILITY 



Project Project Logic Project Plausibility 



Alabama Aviaticn College 


Low 


High 


Ben Hill Institute 


Medium 


Medium 


Bronx Connmunlty College 


Medium 


High 


Chattanooga State College 


Medium 


Medium 


Clackamus Community College 


High 


High 


Columbia Basin College 


High 


Medium 


El Paso Community College 


Medium 


Medium 


Fox Valley Tech 


Medium 


Medium 


Fresno City College 


Medium 


Medium 


Hampden County Consortium 


Medium 


High 


Home Builders institute 


my 1 1 


ivieQium 


Howard Community College 


Low 


Medium 


Illinois Central College 


High 


High 


Illinois Eastern College 


Medium 


Medium 


Indian Hiiis College 


Low 


High 


John M. Patterson College 


Medium 


Medium 


LTV Steel Co. 


Low 


Medium 


Luzerne Community College 


High 


High 


Nebraska Labor Department 


Medium 


High 


Northampton Community College 


Medium 


Medium 


North Clackamus Schools 


Medium 


Medium 



(Continued on next page) 

5^ 



Table II-5, (Continued) 
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Project Project Logic Project Plausibility 



Partners (PAVE) 


Mediup'. 


High 


Research and Development Center 


High 


High 


Southwestern College District 


Medium 


High 


Valencia Community College (CIM) 


High 


Medium 


Valencia Community College (Health) 


High 


Medium 


Valencia Community College (Tele) 


Medium 


High 


Waubonsee Community College 


Low 


Medium 


West Virginia Department of Education 


Medium 


High 


West Virginia Northern College 


High 


Medium 
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ERIC 



Table II-6 



PREDICTION FOR PROJECT SUCCESS 



Project Logic 



Project Plausibility 



Hich 



Medium 



Low 



High (9) 



Medium (16) 




Low 



(5) 



PH = predicted to be highly successful 
Q = predicted to be potentially successful 
n = predicted to be less successful 
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Table II-7 

ANALYSIS OF PROJECT LOGIC: FY1988 AND FY1989 





High 


Medium 


Low 


Total 


FY1988 


8 


10 


5 


23 


FY1989 


9 


16 


5 


30 



Table II-8 

ANALYSIS OF PROJECT PLAUSIBILITY: FY1988 AND FY1989 





High 


Medium 


Low 


Total 


FY 1988 


4 


12 


7 


23 


FY1989 


13 


17 


0 


30 



Table II-9 

RANKING FOR LOGIC AND PLAUSIBILITY: FY1988 AND FY1989 





Successful 


Potentially 
Successful 


Less 
Successful 


Total 


FY1988 


3 


10 


10 


23 


FY1989 


4 


21 


5 


30 
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As shown in Table 11-10, FY1988 and FY1989 projects were deemed 
logical at about the same rates, with the exception of condition five 
(linkages than gaps between short-term and long-term outcomes). Fewer 
FY1989 than FY1988 projects showed more linkages than gaps between 
short-term and long-term outcomes. 

In conclusion, the rate at which projects were predicted to be 
successful doubled from FY1988 to FY1989. This increase was linked, in 
part, to relaxing the condition that projects demonstrate public/pri- 
vate partnerships in scoring plausibility. Projects from both years 
were rated about the same with respect to logic, while FY1989 projects 
tended to score higher in plausibility. 

Recommendations to Strengthen Applications 

To strengthen grant applications—as well as the review process— 
the Office of Vocational and Adult Education could require a project 
flow diagram (see Figure II-3) as part of each applicant's "Plan of 
Operation." The following instructions could be given applicants: 



Instructions for Writing the Plan of 
Operation : State the major objectives of 
your project. These should be entered under 
"Final Outcomes" in the fifth column on the 
flow diagram. Next, list the resources in 
the first column (including Federal and non- 
Federal) used to support the project. State 
dollar amounts. 



Under the activities column, list activities 
to be undertaken at the beginning of the 
project. Start each item with an action 
verb, e.g., "convene the advisory panel," 
"hire instructors," "assemble equipment." 
Then draw a line linking each resource with 
the appropriate activity. This shows which 
resources are allocated to each activity. 



Go 
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Table 11-10 
SCORING PROJECT LOGIC: FY1988 AND FY1989 



Conditions 


FY1988 


FY1989 


One or more long-term outcomes 


23 


30 


One or more short-term outcomes linked 
with long-term outcomes 


10 


11 


One or more activities linked with short- 
term outcomes 


21 


29 


More linkages than gaps between 
activities and short-term outcomes 


16 


23 


More linkages than gaps between short- 
term and long-term outcomes 


5 


2 
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In the third column, list short-term outcomes 
resulting from each activity; e.g., an 
outcome from "convene advisory panel" might 
be "draft recommendations of skills to be 
taught." Second short-term outcome might be 
"incorporate skills into training module." 

Draw a line between each activity resulting a 
short-term outcome. Link each short-term 
outcome. Example: 

Resource Activity Short-term outcome 

Federal > hire > develop curriculum 

$300,000 instructors 



The diagram is complete if all short-term 
objectives lead to one of the long-term 
outcomes listed in column five. Draw a line 
linking short-term outcomes leading to 
separate long-term outcomes. If any short- 
term outcome does not directly relate to one 
of the long-term outcomes, it may not belong 
in the project. Base the "Plan of Operation" 
in the narrative section of the application 
using the flow diagram. Don't forget to 
include the flow diagram with the Plan of 
Operation . 



Other Federal grant programs, e.g., demonstration grants for 
preventing alcohol and other drug abuse, have required applicants to 
include logic models as a formal part of the application. The 
instructions given to their applicants are as follows: 



A logic model should underlie the conceptual 
development and preparation of the proposal. The 
logic model is a conceptual framework that links 
(1) basic assumptions about risk/protective 
factors, (2) mechanisms of intervention and (3) 
outcomes. In this model, the application: 

- specifies the risk factors and/or 
resiliency/protective factors at the 
individual, parent and family, peer 
group, school or neighborhood levels 
being addressed in the proposal; 
articulates the assumptions being 
made, that is, relates these risk or 
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resiliency/protective factors to the 
use and abuse of alcohol and other 
drugs by youth; literature documen- 
tation; 

- relates the proposed strategies or 
interventions to specific risk/pro- 
tective factors, that is, explains 
why these interventions should help 
reduce these risk factors or enhance 
these resiliency factors; and 

- Identifies the measurable/observable 
outcomes that can logically be 
anticipated from the interventions. 



OVAE may wish to include a similar requirement in future demonstration 
regulations. 



So 



Section III 
PROJECT IMPLEMENTATION 



III-l 

in. PROJECT IMPLEMENTATION 



A. Framework for Studying Implementation 



The analysis of project implementation was based on a plan 
developed early in the study but later revised. In the original plan, 
the study team concentrated on the following set of questions: 



Project Administration : 

■ Did the project establish the administrative 
infrastructure necessary for its activities, 
i.e., di' the project hire staff, obtain 
space, schedule services, engage an 
evaluator, etc.? To what extent was the 
project infrastructure already in existence 
when the grant began? 

■ What kinds of partnerships were established 
and what problems arose in creating them? 
Was public-private cooperation established, 
and what other partnerships arose (with 
support service providers, service delivery 
areas, etc.)? 

■ What did projects do to prepare for training 
or other direct services, e.g., skill 
identification, curriculum development, 
licensure, coordination with other providers, 
or staff training? 

■ What was done to recruit students? In 
particular, what type of students did 
projects seek to attract, and how did the 
projects go about doing it? Were there 
particular efforts to attract special 
populations, and, if so, which populations? 

■ What was done to continue the project beyond 
the 18 months of the Federal grant? If the 
project was continued, were the activities 
successful? 



Project Content and Intermediate Outcomes : 

■ What kinds and amounts of training or other 
direct services were provided to students 
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through the grants? How many were trained? 
How many got jobs? 

■ How did projects identify needed job skills 
and how was the information used? Project 
proposals suggested substantial amounts of 
time for skill identification—hence the 
evaluation sought to determine the use of the 
skill information. 

■ What curricula were developed and how were 
they used? As with skill identification, 
curriculum development was emphasized in 
grant applications. 

■ What linkages were developed with other 
public sector agencies or community-based 
organizations? With what effects? To what 
extent did other agencies aid in recruitment, 
training, placement, etc.? 



Extent and Kinds of Innovation : 

■ To what extent did the grants provide 
opportunities for new directions and 
experimentation? 



After analyzing the FY1988 grantees and incorporating the 
suggestions of the advisory panel and Federal officials, the study team 
revised the evaluation plan. Although efforts to characterize the 
projects along the first two dimensions outlined above were retained, 
the implementation analysis was expanded to focus on five issues: 



■ Nature of the training; 

■ Type and intensity of public-private 
cooperation; 

■ Meaning of "high technology" within the 
projects; 

■ Extent to which the projects were 
operationally successful; and 

■ "Exportability" to others of what was 
developed or learned at each site. 
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The final item, exportabil ity, substituted for the examination of 
innovation that was originally planned. The Federal office 
administering the program argued that it was not a condition of this 
demonstration program that projects be new, innovative, or 
experimental. Hence, assessing projects on the extent of their 
innovati veness would be unfair. Each of the five issues is briefly 
discussed below. 

The Nature of Training . The evaluation of FY1988 grantees showed 
that the types and amounts of training delivered under the program 
differed considerably among the projects. The range of training 
activities was impressi ve--from half-day workshops for business 
managers on the virtues of a particular piece of equipment to two-year 
technical degree programs for nontraditional students. The range also 
meant, however, that it was hard to discern the precise nature of the 
federal demonstration, to understand if there were sufficient 
commonalities among the projects to describe them jointly, and to 
conduct the cost-benefit analysis which was a mandated part of the 
evaluation. 

There was also a specific question raised in the Year One report 
about the efficacy of customized training as a part of the 
demonstration effort. Three of the eight Year One high-tech grantees 
visited by the evaluation team were customized training projects. The 
grantees—usually community colleges— provided highly specific training 
to the employees of individual companies. Often the training was 
designed to enable the employees to use a piece of specialized 
equipment or software, although it might also be aimed at teaching more 
generic occupational or basic skills. The regulations of the 
Cooperative Demonstration Program note that all projects assisted under 
the program must be capable of wide replication by service providers-- 
an outcome that seems questionable for many customized training 
projects. Questions were raised in the Year One report about both the 
overall desirability of narrow training under Federal sponsorship, as 
well as about the specific priorities that could be introduced to 
ensure that some broader public interest was served. 
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The advisory panel also raised questions about the advisability of 
supporting demonstration projects that provided students with short- 
term training aimed at specific, entry-level jobs (regardless of 
whether it was customized for a particular employer). This form of 
training was offered by many of the Year One grantees visited. The 
panelists argued that demonstrations should be concerned with how to 
increase trainee skills and employment options, not just with the 
demands of specific entry-level jobs. There was considerable interest 
in whether projects supported through the program were engaged in 
public-private cooperative ventures to deliver both job-specific and 
more general skill training and what could be learned from such 
ventures. 

The Type and Intensity of Public-Private Cooperation . Based on 
Year One findings, the evaluation team identified a variety of public- 
private relationships: 

■ The private sector partner was a customer of 
the project--the project provides training 
for employees of the company; 

■ The partner was a member of an advisory 
committee or an informal board; 

■ The partner was a supplier of resources such 
as equipment, and/or jobs at project 
completion; 

■ The partner was an active participant in 
delivery of instruction or other services; or 

■ Ihe partner initiated the project and 
approached the grantee for assistance. 

Among Year One grantees, most of the projects visited had 
relationships with private sector partners in which the partner played 
a customer or advisor role, sometimes also contributing equipment for 
training. A few projects did not have private sector businesses as 
partners. In only one of the eight grantees visited did the evaluation 
team see a project in which a private sector business had participated 
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actively in both the design of the project and in service delivery. As 
with the search for interesting training models, then, the desire in 
Year Two was to identify partnerships that might be breaking new 
ground, i.e., that might provide models of public-private cooperation 
for high-tech training. 

Defining High Technology . Based on Year One findings, the meaning 
of high technology, like the meaning of publ ic~private cooperation, 
varied considerably. As discussed in Section I, the definition 
included in the law and regulations was quite broad. Based on Year One 
findings, the study team characterized the actual high-tech elements of 
the projects the study team visited first in terms of the field for 
which students were preparing. If the field was not high-tech, the 
study team then focused on the specific training received. Four 
definitions of high technology emerged: 



■ Training was designed to prepare students for 
jobs in fields generally considered high-tech 
because they manufactured (or serviced) high 
technology products. The training itself may 
have been sophisticated or basic, and have 
led to upgrading skills of current workers or 
to entry-level jobs. 

■ Training enabled students to use high-tech 
equipment even though the field in which they 
worked (or sought work) was not considered 
high tech. For example, one project taught 
computer assisted design (CAD) or the use of 
sophisticated diagnostic equipment to repair 
automobiles. 

■ Training taught students to use computers, 
irrespective of the field for which training 
occurred. For example, some projects 
prepared students to be secretaries by 
teaching word processing skills. 

■ Training was provided in basic skills as 
preparation for later occupational training 
in a high-tech field. The grant supported 
the basic skill phase of instruction with the 
understanding that the students who improved 
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their basic skills would enter training that 
met one of the three definitions above. 

In short, not all projects shared a common understanding of what was 
meant by high-tech training. To identify the range of projects, 
especially those that were training students in ways that appeared to 
be "state of the art," the second year of the evaluation focused on 
understanding the high-tech elements of the projects studied. 

Operation and Integration of the Demonstration within the 
Institution . In Year One there were a number of projects that 
experienced delays in start-up and appeared to be in danger of 
terminating with the end of the Federal grant. The study team 
identified timing issues that appeared to affect start-up, but reasoned 
that projects with stronger ties to the institutions in which they were 
located might have a better chance to start promptly and continue after 
the end of the grant period. They might also be better able to draw 
upon the expertise of regular teaching staff and use facilities 
effectively in program development and delivery. This section looks at 
this and other problems that arose in implementing project designs. 

Exportabilitv of Project Design, Activities, and Products . The 
Cooperative Demonstration Program was intended to demonstrate models of 
public-private cooperation in the delivery of training. As stated in 
the regulation, all projects were to be "(1) Of direct service to the 
individuals enrolled; and (2) Capable of wide replication by service 
providers" (CFR 34, Ch. IV (7-1-88 Edition), Part 412, Subpart B). The 
program was to produce 

model projects providing improved access to quality 
vocational education for [special populations] and 
...examples of successful cooperative between the 
private sector... and public agencies in vocational 
education. . . 

The way in which the study team chose to characterize the demonstration 
element of the Cooperative Demonstration Program was exportabil ity , 
i.e., that which can be taken from the projects that have been 
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supported and applied elsewhere. In other terms, what can a service 
provider interested in effective partnership use or apply from what has 
been produced as a result of the Cooperative Demonstration Program? 

Some projects were clearly demonstrations by design. These were 
projects aimed at development o? a specific product, usually a 
curriculum, that can be applied by others. They used the grant to 
develop the curriculum and conduct a formative evaluation (pilot test) 
or they used the grant to test the repl icabil ity of the curriculum in 
additional institutions or with new populations. These projects were 
usually characterized by formal evaluation designs that accompanied 
project (curriculum) development and implementation. In Year One, only 
a very few of the projects were of this type. 

More commonly, projects were aimed at adding dimensions to, or 
improving the delivery of, training at the grantee institution. While 
that was certainly a worthwhile goal from the institution's 
perspective, it does not necessarily translate into a demonstration 
unless the process of institutional "capacity building" yields two 
things. First, it must provide evidence of effectiveness--i .e. , that 
the procedures undertaken did, in fact, build the institution's 
offerings or improve the delivery of instruction in measurable ways. 
Second, it must yield a guide or other device that shows others how 
they can accomplish the same changes using the same procedures--i .e. , 
it must be made capable of replication. The study team sought to 
determine which, among the FY1989 projects, held promise as models 
capable of replication. 

The implementation analysis in this section is organized around 
these five issues. If the reader would like additional information on 
project administration, content, and intermediate outcomes, please 
refer to the report on the overall findings of the implementation 
survey in Appendix F. Additional information, including more detailed 
grantee-by grantee-descriptions of project activities, partners, 
clientele, staffing, and dissemination can also be found in the matrix 
of projects at the end of this section. 
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B. Procedures for Studying Implementation 



The implementation analysis included four data collection steps. 
The first two steps were carried out with FY1988 grantees. Shortly 
after the evaluation began (March 1990) the evaluation staff refined a 
set of implementation issues outlined in its technical proposal, 
selected nine Year One grantees for further study (based on a 
preliminary evaluability assessment), and conducted a telephone survey 
of the nine grantees (April, 1990). Immediately after the telephone 
survey, the team conducted visits to eight of the nine grantees. All 
the visits were completed by the end of June, 1990, when most of the 
Year One projects ended. 

These two data collection steps (telephone survey and visits to 
eight projects) yielded data that were the basis for the implementation 
portion of the Year One report. They also provided the evaluation team 
with the opportunity to reflect on the original design and make some 
minor modifications. These changes were discussed earlier in this 
chapter. 

The two data collection efforts in Year Two were considerably more 
ambitious. After 0MB clearance, a systematic mail survey was mounted 
with questionnaires distributed to all 30 grantees. Despite initial 
mailings by the OVAE program managers and extensive telephone follow 
ups, only 26 grantees (87 percent) returned questionnaires. (Results 
are reported for 27 grantees, however, because one grant yielded two 
sub~grants with unrelated projects.) Names of respondents are included 
in Appendix F, the report of survey findings. The survey was followed 
by two-day visits to 19 of the 30 FY1989 grantees. 

The site visit teams used a similar protocol to the one used in 
Year One, although a few changes were made during the 0MB clearance 
procedure. As in the first year, the study team prepared site visit 
reports. The site visit information was then condensed into three- to 
four-page descriptions (see Appendix E). Based on the descriptions and 
discussions with site visit teams, the study team created the data 
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displays at the end of this section summarizing the qualitative data 
obtained from each site. 

The Year Two data collection efforts form the basis for the 
results reported in this chapter. Specific information from the mail 
survey is referenced in the text. For some issues, the study team 
relied more heavily on site vi.^it information. For others, the study 
team relied primarily on information derived from the survey. The 
study team tried, whenever possible, to supplement one with the other. 
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C. Major Findings from the Implementation Study 



1> The Nature and Breadth of Training 

There are several dimensions along which the training efforts of 
FY1989 projects may be discussed. One dimension is simply the extent 
of training, i.e., the number of hours, weeks, or months that students 
typically participate. A second dimension is the content of the 
training—is it detailed, job- or equipment- specific instruction, an 
overview or introduction to a field with generic as well as more 
specialized information, or is it refresher basic skills instruction in 
preparation for occupationally-specific instruction later. Individual 
Cooperative Demonstration projects reflect all of these kinds of 
training, and several projects combine several different kinds of 
training. 

The amount and content of training is also related to other 
factors, the most important of which is probably the clientele. 
Specialized instruction may be more appropriately delivered to some 
clients than others, notably to persons who are already engaged in 
occupations akin to the area of instruction. 

Amount of Training . The FY1989 projects that were visited offered 
relatively short educational programs. Although the designs of the 
various projects ranged from workshops of less than a day to two-year 
programs yielding associate degrees, the majority of projects developed 
short-term offerings. Many of the projects provided training to more 
than one clientele or for more than one type of job, with the amounts 
of time varying by clientele and position. 

The shortest training efforts were essentially workshops, i.e., 
programs of one day or less. These were commonly offered to business 
owners or employees, often to demonstrate new technologies or 
equipment. For example, one project provided weekly two-hour 
demonstrations of computer integrated manufacturing (CIM). Employers 
then signed up for short-term training in computer assisted drafting 
(CAD), computer assisted manufacturing (CAM) and other topics with 
training usually lasting one day. Another project offered four-hour 
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workshops on disk operating systems (DOS), CAD and CIM. A third 
project introduced large numbers of employees in one company to a new 
data collection system, training them in three-hour sessions before 
work, followed by two weeks of on-the-job training (OJT). 

Several projects were designed to offer 100 to 200 hours of 
instruction spread over a six-month period. For example, one project 
taught students a computer language, providing 108 hours over six 
months. Another project offered a series of semester-length courses in 
CAD and programmable logic computers. Two projects offered entry-level 
training programs for health-care technicians of less than six months 
duration. One project provided students a minimum of 12 hours a week 
of training in business and office skills, with students remaining 
until placed in jobs— typically about four months. 

Another set of projects offered training lasting a year or less. 
These projects tended to offer a license or certificate at completion. 
One project created an 11-month program to train respiratory therapy 
technicians, while a second health-related project trained students as 
technicians in a variety of specialties through one-year programs. 
Another project established a six- (later eight) week overview and 
nine-month certificate program in precision metal fabrication and screw 
machine technology. One project trained disadvantaged adults in 
desktop publishing during a 400-hour program, while another developed 
the a one-year program in software applications. 

A final set of training projects was designed as two-year programs 
leading to technical degrees or licenses. For example, one project 
designed a two-year training program in hazardous waste management. 
Another was initially designed as an IS month training effort for 
radiology technicians but will be expanded to 24 months in the future. 
The one FY13S9 project aimed at high school students created a two-year 
occupational "clusters" program with internships. Two grants supported 
institutional capacity-building efforts, with one school developing an 
associate program in advanced technology and the other revising an 
existing associate-level industrial technology program. 
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Projects aimed primarily at curriculum or product development are 
harder to characterize using the time dimension. One project developed 
a curriculum to train entry-level employees in semiconductor 
manufacturing over approximately 283 hours. The training portion of 
this project was intended to pilot test the product. Another project 
developed an 18-hour program for master electricians to enable them to 
install home automation systems. Three projects developed one or more 
videodiscs to be used in conjunction with ongoing training programs. 

The relatively short duration of training observed in the 19 site 
visits was reflected by the findings of the survey of FY1989 grantees. 
Grantees reported that 68 percent of participants received between one 
and 100 hours of training, while 11 percent received 101 to 250 hours. 
Approximately 14 percent of participants received 1,000 hours of 
training or more. 

There is a major caveat to note with respect to the hour totals, 
however—one that suggests that the actual number of grant-supported 
hours of training may have been lower than these numbers reflect. Not 
all projects reported total instructional hours received through the 
grant in the same manner. The questionnaire instructions asked for 
"hours of instruction. . .from the project during the grant period," but 
it was likely that at least some projects included total instructional 
hours received by participants, whether or not the- training was project 
supported. For example, a project may have provided revised or 
additional services to an ongoing course or degree program. Some sites 
reported the total number of hours required to complete that course or 
degree program as the instructional hours received under the grant. 

Training Content . As with the duration of training, there was 
considerable range in content areas. Most of the projects that 
received site visits offered training in three broad areas — 
manufacturing, health, or business/office. Projects in manufacturing 
appear to have been focused on both entry-level and more advanced 
skills, while the health and business/office projects were focused on 
the development of entry-level skills. 
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Projects offering manufacturing-oriented training tended to train 
participants to use similar types of high technology equipment. At 
leaSi: five of the projects taught students computer-assisted 
manufacturing applications including the use of CAD, CAM, computer 
numerical controlled (CNC) machining or programmable logic controllers 
(PLCs). An additional project taught the application of a specialized 
computerized manufacturing system for a particular plant. These 
projects tended to be short-term. Some began with demonstrations of 
the use of the equipment, followed by short classes for persons seeking 
to use the systems. Almost all the students in these projects were 
current employees of companies. The one exception was a two-year 
program in the use of CNCs aimed at women seeking work in manufacturing 
(although the majority of students in the program were men). 

A few manufacturing projects receiving site visits did not fit the 
short-term/CAD~CAM model. These included a 48-month apprenticeship 
program for Electronic Instrument Repair Technicians (EIRT), an ongoing 
program in which the Cooperative Demonstration grant supported an 18- 
month portion; and the nine-month certificate programs in precision 
metal fabrication and screw machine technology. The model may also not 
fit precisely a project that trained pipefitters and millwrights to use 
a new, automated equipment system at a steel mill. The EIRT program 
and pipefitter/millwright programs were two of the most customized 
training efforts, geared to the specific machinery of the partner 
companies. Another project offered a wide range of entry-level 
programs for disadvantaged students, including a 254-hour oasic 
machining course. Finally, one manufacturing project revised and 
expanded the offerings of an industrial technology program at a 
community college. 

The health projects were generally of longer duration and were 
geared to entry-level technician positions. For example, one project 
trained students as licensed practical nurses (LPNs) and alcohol/drug 
counselors. It provided a year of specialized training and OJT 
combined with remedial basic skills. Another project offered both a 
5.5-month training program for pharmacy technicians and an 11-month 
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training program for respiratory therapy technicians. The longest such 
program for adults was 18 months and trained participants to be 
radiology technicians. All of these projects had a clinical component. 

There were two health projects that did not reflect the health 
technician training profile. One developed interactive videodiscs that 
taught viewers to use volumetric pumps. It was designed to be used by 
students already familiar with intravenous fluid delivery and was 
tested in a refresher course for returning nurses. The other project 
cut across the health and business sectors, training disadvantaged 
persons for entry-level work in the health field, although much of the 
training was for clerical positions (i.e., medical transcription, 
medical records apprentice, medical unit clerk). Students in this 
project received an average of 120 to 140 hours of instruction. 

In general, business/office projects that received site visits 
tended to focus on entry-level skill training, although the hours of 
training varied widely. Two projects were extensions of Job Training 
Partnership Act (JTPA) training programs. One provided a 400-hour 
training program in desktop publishing (as well as other, non-business 
offerings), while the other offered a set of 40 self-paced office 
technology courses. Students with prior typing skills also received 
unpaid work experience, and it was estimated that the typical student 
spent aboiit four months in the program before finding employment. The 
high school office systems occupational cluster program was two years 
in length, with an internship of four to six weeks. Finally, one 
business grantee designed several different options, including a one- 
year program leading to a certificate in microcomputer software 
appl ications. 

There were a few projects that included training in fields other 
than manufacturing, business, and health. As already noted, one 
project developed an 18-hour curriculum to enable master electricians 
to install home automation systems. In addition, one of the JTPA 
programs that offered a wide range of courses offered a 254-hour 
program in automobile repair. Two projects were focused on computer- 
related training, with one developing a six-month program in ADA 
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computer language and another "packaging" a number of courses to create 
a two-year advanced technology degree. 

Only a few of the projects that received site visits spent large 
amounts of instructional time on non-occupational training. The most 
common form was remedial basic skills instruction, which was provided 
in health projects and others geared to disadvantaged students. For 
example, the project providing training for medical/clerical entry- 
level work also provided students with a self-paced, computer-based, 
remedial skills program. The project to train LPNs and drug/alcohol 
counselors reported that it provided a total of 2,000 hours of remedial 
education. Also providing basic skills instruction were the 
apprenticeship project for EIRTs and the project to train entry-level 
workers in manufacture of precision metals. It was likely that more 
advanced academic skill instruction was included in the various 
certificate and associate level programs. 

The relationship between industry and duration of training was 
reflected in findings from the survey of grantees. Respondents 
identified the industries in which they trained, and each response was 
then compared with the hours of training reported for participants in 
that project. Most of the students (82 percent) in projects that 
indicated manufacturing as an area of training received less than 100 
hours of instruction (see Table III-l). This compares with 43 percent 
of students in business service projects and six percent in health care 
projects. Overall, health care projects delivered the largest number 
of hours of training, with most students (80 percent) receiving 250 
hours or more. 

In addition to occupationally specific training, a number of 
projects offered basic skills or other related instruction. According 
to questionnaire responses, the majority of the projects (16 of 27) 
provided non-occupationally-spacific training. The most common forms 
of non-occupationally-specif ic training were provision of employabil ity 
skills (13 projects), provision of basic academic skills, (11 
projects), and provision of advanced academic skills (9 projects). 
Interestingly, there was little relationship between the field of 
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training or hours of training and the likelihood of non-occupational 
instruction, but projects geared to disadvantaged students were more 
likely to provide non-occupational training. 

Clientele. Observations derived from site visits were reflected 
in survey findings as well. According to the survey, projects 
providing training in manufacturing were the most likely to indicate 
current employees as their primary clientele type. As shown in Table 
III-2, ten of 17 such projects indicated that current employees were 
their primary clients. Projects providing training in multiple 
industries also commonly indicated that they served existing employees 
(four of seven projects). Projects providing health care training were 
more likely to indicate that they served adults seeking work in a field 
or company (three of seven). Only one project providing health care 
indicated that it served current employees of companies as its main 
cl ients. 

Projects offering training in business services were somewhat .nore 
likely than others to indicated that they served disadvantaged adults 
(two of five projects). Projects training for manufacturing and 
multiple industries were also more likely to be training persons 
employed full-time (see Table 1 1 1-3). Although information on 
employment status of clientele was not available for all such projects, 
of the 16 projects training students in manufacturing for which data 
exist, seven indicated that 76 to 100 percent of students were employed 
full time, and an additional three indicated full time employment by 51 
to 75 percent of students. In contrast, none of the six projects 
providing training in health care for which information was available 
indicated the 76 to 100 percent of students were working full time, and 
only one indicated that 51 to 75 percent were working full time. 

Summary . Overall, projects provided relatively short-term 
training. Survey results indicated that most participants received 
training of less than 100 hours. Projects included in the site visits 
can be grouped loosely into the following three categories: 
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■ Short-term, skill-specific instruction for 
current employees of companies or for persons 
already familiar with the field of training. 
This approach was most common among manufac- 
turing projects, but it could also be seen in 
the refresher nursing course and in business 
projects teaching specific software 
applications, as well as home automation 
installation. These projects rarely provided 
formal credentials. Some instruction was 
customized, tailored to the needs of specific 
employers. 

■ Medium-term, entry-level technician training. 
These were programs in which training ranged 
from a few months to a year, often aimed at 
helping unemployed or otherwise disadvantaged 
persons find jobs in a field. These programs 
cut across the industries but were quite 
common in business. The three programs run 
by JTPA-supported agencies fell within this 
category, one of which provided training in 
several fields. Some provided certificates. 

■ Longer-term programs aimed at two-year 
certification or degrees (associate degree, 
license). Some of these projects were aimed 
at institutional capacity-building and 
regular students; e.g., creating new or 
revising existing offerings within depart- 
ments of industrial technology or business/ 
computers in community colleges. One was an 
apprenticeship training project, and one an 
occupational project for high school 
students. 



Not all 19 projects that received site visits fit neatly into one 
model, and a very few may not fit at all. But most could be identified 
within one or more of these groupings, which provided some sense of the 
range of programs developed and training delivered under the Coopera- 
tive Demonstration Program. 
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2. Type and Intensity of Public/Private Cooperation 

The Cooperative Demonstration Program was designed to encourage 
cooperation between educational institutions and the private sector. 
As the program* s grants announcement noted: 

High technology training can be conducted most 
effectively with the active involvement and 
cooperation of the private sector. Effective 
partnerships between the private sector and public 
agencies in vocational education are an important 
aspect of the Cooperative Demonstration Program... 

The announcement anticipated that the partnerships established by the 
project would provide models of effective cooperation. To understand 
the nature and extent of cooperation, the FY1989 respondents were asked 
numerous questions about their projects' public-private partnerships. 
The following discussion presents the survey results first, followed by 
findings from the site visits. 

Survey Findings . Most projects established relationships with 
employers. Of the 27 respondents, 23 indicated some involvement of 
employers in the planning or administration of the project or in 
providing services. Nineteen projects indicated multiple types of 
partners, including schools or universities (institutions that may 
serve the "private sector" cooperative role or uiay be involved in the 
project in some other manner) and community-based organizations (ten 
projects) . 

Although employers were involved in 23 projects, not all those 
projects indicated that private businesses were the most important 
partner organizations. Only slightly more than half the projects (14 
of 26) indicated that the most important partner was a private 
business. Of the remaining projects, five indicated that the most 
important partner was a trade association or consortium, four indicated 
an educational institution, and three indicated a non-educational 
public agency. For the grantees that had second partners, these 
partners were even less likely to be private businesses (nine of 20) 
and more likely to be trade associations (four of 20). 

Do 
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Most grantees formed partnerships with only a few organizations, 
but some established multiple partnerships. Six projects had one 
private sector partner, and 16 projects had four partners or fewer, but 
seven projects indicated that they had ten or more partners. Given 
this range in the number of partners, it was likely that the roles of 
partners differed considerably across the projects. 

In fact, differences in partner contributions were considerable. 
When asked to rank in importance a list of possible activities for the 
most important partner organization, there was little consensus among 
the grantees. The most commonly selected first choice activity for the 
partner organization was that it provided equipment for training, but 
that choice was selected by only seven grantees. Five grantees 
indicated that the partner's main contribution was identifying job 
skills, while four indicated that the partner recruited students for 
training. Three indicated that the partner served on a project 
advisory committee and two indicated that the partner supplied 
instructors. In short, there was considerable variability in first 
partner contribution— a variability that was sustained when the first 
through third ranked activities were added together. The same findings 
held true for second partners and, in fact, the variability in 
contribution increased. 

The iifferences in activities may be, in part, a function of the 
nature of the partner relationships. Based on categories developed 
from site visits to first year grantees, the evaluation team identified 
several ways to characterize or summarize the relationships between 
grantees and employers. Each FY1989 grantee was asked to pick the 
choice that best characterized its relationship with its first partner. 
The majority of grantees selected one of the following two choices: 

■ The partner was a customer of the project, 
e.g. the project provided customized training 
to the partners' employees (selected by nine 
grantees); or 
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■ The partner shared actively in the delivery 
of instruction and services (selected by nine 
grantees) . 

No other choice was selected by more than three grantees (see Table 
III-4). 

Projects with multiple partners were more likely to report that 
the partners were customers, while projects with few partners used 
partners in more ways. Of the nine projects selecting "customer" as 
the main relationship, five also reported seven or more partners. In 
contrast, eight of the nine projects indicating that partners shared in 
instruction reported six or fewer partners. In addition, projects with 
six or fewer partners reported other uses of partners (advisory 
committee member, resource provider, initiator of project) while none 
of the projects with seven or more partners selected any of these 
categories . 

In most of the projects, the grant did not result in establishment 
of a new primary partnership. Nineteen of 27 grantees reported that 
the relationship between the grantee institution and the partner 
organization was not new, but all 19 indicated that the relationship 
had been strengthened as a result of the project. The eight projects 
that did establish new relationships with the first partner 
organization were more likely to have encountered problems. Three of 
six projects that reported problems with their first partner had new 
partners, and two of these partnerships were dissolved by the end of 
the grant. 

Grantees were slightly more likely to have established new 
partnerships with their second partners. Eight of 21 second partners 
were new, which appeared to reflect a wider range of relationship 
types. Considerably fewer second partners were project custoners 
(two), though a sizeable number (eight) shared in delivery of 
instruction and services. This difference between first and second 
partners may have occurred because second partners were more likely to 
be trade associations or other educational institutions rather than 
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employers. Second partners were somewhat more likely to serve 
primarily as members of advisory committees. 

Site Visit Findings , Among FY1989 grantees that i^eceived site 
visits, partners and their roles ranged widely. In a majority of 
cases, grantee institutions formed partnerships with a small number or 
private companies (or associations) that performed roles such as 
serving on advisory committees, loaning or donating equipment or 
facilities, or helping to identify the skills needed in the jobs for 
which training was provided. This was the case, for example, for the 
college that created a CNC training program aimed at attracting women, 
the college that established an advanced technology center that 
delivered courses to employees of local companies, and the county 
vocational center that taught ADA computer language. But in several 
projects, the study team saw different types of relationships, related 
directly to the way in which the project was organized and the services 
del i^'ered. 

The first type of cooperation was intensive customized training . 
In two projects, the private employer was either the initiator of the 
project or played a central role in design and service delivery. Both 
of these were customized training projects in which the employers were 
steel mills. In one case, the company had already established an 
apprenticeship program for repair technicians (EIRT). It had paid for 
curriculum development by an instructor at a nearby college and needed 
a process control lab for instruction. The company approached the 
college to apply for Cooperative Demonstration funding. As the grant 
could not pay for equipment, it was used to subsidize training and the 
steel company paid for the lab equipment. In the other case, a college 
and a steel company worked jointly to develop training that enabled 
over 1,500 workers to implement new, high-tech milling equipment. The 
company paid for curriculum development and supplied training space, 
OJT, user support teams, plant floor support teams, and vacation 
reimbursement for the time workers devoted to training before work 
hours. 
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Other projects reflected the customized training model in somewhat 
less intensive form. Several projects began by trying to enlist 
students through convincing employers that training could benefit them 
and their workers. Projects staff then worked with individual 
employers to develop instruction tailored to the needs of their 
companies, and employees received short-term training. In these cases 
the training was not quite as customized (CAD, CAM, etc.). Most of 
these projects hoped to continue after the grant by generating fees 
from the businesses for the same or similar forms of instruction. 

The second type of relation ship was recruitment by the grantee 
and referral to the partner for training . In these cases the grantees- 
-some of which were administerea by agencies that receive ongoing 
support From JTPA— were community-based agencies {or other entities) 
that recruited participants. They might also assess the students* 
skills before referring them to another institution for training. They 
viewed their primary partners as being the colleges, technical 
institutes, or other institutions (such as hospitals) that provided the 
actual training. These grantees tended to view the private sector 
primarily as potential employers. Potential employers, such as welfare 
agencies, might also refer persons to the grantees. These projects 
tended not to have advisory committees of employers, but stayed in 
contact through efforts to place students in jobs. One such project 
described itself as "managing" a partnership of training institutions 
and potential employers. 

The third type of relationship occurred in projects focused on 
curriculum development . The projects used schools or other 
instructional settings to "test" their offerings, and private sector 
partners (businesses or associations of businesses) as ongoing 
reviewers of materials. Some also used the employers to obtain initial 
information on skills that was then incorporated into curriculum 
development. The projects aimed at instruction in semiconductor 
manufacture, manufacture of precision metals, use of volumetric pumps, 
development of high school occupational clusters, and installation of 
home automation operated in this manner. Because the high school and 
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volumetric pump projects also involved internships, the employers also 
supervised the students who were piloting the approach and provided 
feedback. 

Summary . In many projects, then, the nature and organization of 
project activity was linked to the type of partnership that arose. 
Among projects included in site visits, intensities of partnerships 
varied from the two projects where a single employer played a critical 
role in service development and delivery, to projects in which 
employers might do little more than attend meetings of an advisory 
committee. Overall, intensity of partnership may be less significant 
than number of partners in determining partner role. Survey results 
show that projects with multiple partners wpre considerably more likely 
to report that partners were customers, while projects with few 
partners used partners in more ways, including active sharing of 
instruction. 

3. The Meaning of High Technology 

The projects also reflected different interpretations of the 
Congressional mandate that projects involve high technology. As 
indicated previously, the evaluation identified four basic ways in 
which FY1988 projects incorporated high technology in developing their 
prograins and delivering training. Projects were high-tech in that: 

■ Training was designed to prepare students for 
jobs in fields that manufactured high-tech 
products or serviced high-tech equipment; 

■ Training was designed to enable students to 
use high-tech equipment or products even 
though the field in which the equipment was 
used was not generally considered high-tech; 

■ Training was conducted on high-tech 
equipment, such as computers, CAD, or CIM 
equipment; or 

■ Training was offered in basic skills as 
preparation for specific occupational 
training in a high-tech field. 
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Survey Findings . FY1989 survey respondents were asked to select 
all the high-tech definitions above that were appropriate to their 
projects. Most respondents selected more than one response to describe 
their projects' high-tech elements. Therefore, the study team created 
one response category for each project for further analysis: 

■ Training for high-tech field~ll respondents; 

■ Training for high-tech (equipment in non-high- 
tech field—nine respondents; 

■ Training for both high-tech field and for 
using high-tech equipment in a non-high-tech 
field— three respondents; 

■ Training on high-tech equipment--one 
respondent; 

■ Basic skills training to prepare for further 
training in a high-tech field--two 
respondents; or 

■ Basic skills training utilizing high-tech 
equipment in preparation for further training 
in a high-tech field--one respondent. 

Using these categories the study team first observed that most of 
the projects (23) defined "high-tech" in terms of the occupation or 
equipment for which training was provided. Of the 23, about half (11) 
were explicitly preparing students for immediate or specific work in 
high-tech fields. Most of the others (nine) were preparing students to 
use high-tech equipment or products in a non-high-tech field, with the 
rest doing both. If these answers were indicative, the remaining four 
projects were not preparing students for explicit high-tech applica- 
tions, but rather defined their pv^ojects' high-tech element in terms of 
the equipment used in training or the students' long range occupational 
goal . 

Second, the study team examined the extent to which the 23 
projects that defined their high-tech focus in terms of occupation or 
field application also used high-tech equipment or provided basic 
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skills instruction as part of their activity. The study team found 
that of the projects that prepared students for work in high-tech 
fields, the vast majority (eight of 11) used high-tech equipment in 
training (see Table III~5). Among the projects preparing students to 
use high-tech equipment in non-high-tech fields, iiowever, only a slim 
majority (five of nine) conducted training on high-tech equipment. 
Since these projects were explicitly preparing students to use high- 
tech equipment or products on the job, the lack of training on such 
equipment in a majority of them was surprising. It suggests that the 
training may have been quite limited in scope. 

The projects preparing students for ui\ng high-tech equipment in 
non-high-tech settings were also more likely not to offer basic skills 
instruction. Only two of the nine such projects indicated basic skills 
instruction, compared with six of the 11 projects preparing students 
for jobs in high-tech fields. This finding also suggests that the 
training for use of high-tech equipment in non-high-tech fields may 
have been rather limited or narrow in scope. 

Projects that included manufacturing as an area of training were 
considerably more likely than others to prepare students for work in 
high-tech fields. Eleven of 17 such projects (65 percent) prepared 
students for high-tech fields (see Table III-6). Projects described as 
training for multiple industries also were weighted toward preparation 
for high-tech fields, with four of seven selecting this choice. This 
compares with only one of seven projects that provided preparation for 
health care jobs. None of the four projects that included training for 
the transportation industry indicated it was training for a high-tech 
field, and business service projects were about equally divided between 
high-tech and non-high-tech fields. 

Site Visit Findings . The site visit results parallel those of the 
survey, although sites that were visited tended to do more training for 
use of high-tech equipment. Most of the projects were preparing 
students to use high-tech equipment on the job, whether or not the 
field for which training was conducted was itself high-tech (see matrix 
of projects at the end of this chapter). Many of the projects also 
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used high-tech equipment, especially computers, in the course of 
instruction, but only one project appeared to meet that high-tech 
criterion alone. 

Initial concerns that projects might not nave a high-tech focus 
did not appear to be have been reflected in practice. This was 
primarily true because almost all fields have some high-tech elements. 
If a business project was training students to use word processing it 
was teaching students to use "high tech" equipment. The same was true 
for an auto mechanics project that was teaching students to use 
sophisticated diagnostic equipment. There was almost no field in which 
certain high-tech elements did not exist. Because the Cooperative 
Demonstration Program made no distinction between operating equipment 
and understanding how it functions, the requirement that the project be 
high-tech in focus excluded virtually no training effort. 

4. Operation and Integration of the Demonstration within the 
Institution 

There were numerous reasons to examine whether projects were able 
to start and operate effectively, and whether they were eventually 
integrated into theiv^ institutions. Because this was a demonstration 
program, the regulations envisioned wide replication of project 
efforts. For a project to be replicable, however, it must be suffi- 
ciently active (or its products available) for others to adopt it after 
Federal funding ends. In addition, because the grant period was short 
there had been concern that 18 months might not be sufficient for the 
projects to start, complete their activities, and generate continued 
support. In Year One, several of the projects experienced start-up 
problems that delayed services and made completion of objectives 
difficult. As a result, the study team asked a number of questions 
designed to examine start up, project operation and continuation beyond 
Federal funding. 

There appeared to have been fewer start-up problems among the 
FY1989 grantees than among FY1988 grantees, but problems remained. 
Problems were generally of three types— those that were primarily 
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administrative, those that were the result of incomplete or unworkable 
plans for the projects, and those that resulted from the economic 
downturn that affected some communities and sectors of the economy. 

Administrative Issues . According to survey results, most of the 
FY1989 grantees said their projects would not have existed without 
Federal support. Of the 27 projects, only four indicated that the 
project would have existed without the Federal funds. Three of those 
four were providing grant-related services prior to the start of the 
grant. At the completion of the grant period, 14 projects expected 
that they would continue in their entirety, with another eight 
indicating that they would continue in a scaled-down form. Two 
projects indicated that their Federal funding had not yet lapsed. 

Projects generally began to provide direct services to clients 
within a few months after the start of their grants, but a few did not 
provide services for many months. Excluding the three projects that 
were providing service to clients before the grant began, 12 projects 
began providing services within three months of grant award, and nine 
more began within nine months. Three projects did not start, however, 
until ten to 12 months after the grant awards (or six to eight months 
before the grants were originally due to end!). 

Nonetheless, the grant period appears to have been sufficient for 
many of the projects to become institutionalized or find other sources 
of support. Twenty-two projects indicated they continued beyond the 
end of the grant. The most common sources of support among continuing 
projects were funds from the grantee institution (11 respondents), with 
private employers the second most common (eight respondents had 
commitments from employers and four others were waiting for final 
agreements). Five respondents planned to charge students tuition for 
project services. Of course, students were also likely to pay regular 
college tuition in projects indicating they would continue with 
institutional support. 

Interestingly, the ability of projects to provide services soon 
after the grant award appeared to make little or no difference in 
whether they continued services beyond the grant period (see Table III- 
7). In fact, there was a slightly greater tendency for projects that 
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began providing services last to continue, with all of the latest- 
starting projects continuing in their entirety after the completion of 
the grant. Two of the three projects that indicated they were 
providing services before the grant started either received extensions 
to December, 1991, or were no longer providing services at the time of 
the survey. 

Yet, when asked directly about factors affecting their ability to 
complete their original plans, the short grant period was cited as 
problematic by the largest number of respondents. Nine projects 
indicated that they encountered some problems in completing their 
original plans. The most commonly cited reason for implementation 
probleiTis was that the 18-month grant period was insufficient, a reason 
cited by five of these nine projects. Also noted by more than one 
project were difficulties in staff recruitment and/or retention (two 
projects) and planned activities that proved inappropriate (two 
projects) . 

At the same time, however, ten projects indicated that they had 
accomplished activities they had not origir.ally planned. The most 
commonly cited additional activity was curriculum development (five 
projects), followed by partner recruitment (four projects), and 
dissemination (three projects). Over half of these projects (six) 
indicated that they obtained additional funds that enabled them to 
undertake these unplanned activities. 

During site visits, the details of several administrative problems 
were apparent. At least three projects encountered some difficulty 
finding project directors or other staff. The reason was usually the 
same— it was hard to find talented persons willing to take on 18-month, 
closed-end positions. In some cases the situation was complicated 
further because project positions were temporary or part-time and, 
hence, did not include benefits. Overall, however, these problems may 
have occurred less often in the second year because the recession may 
have made people more willing to take less attractive positions (and 
also because several p^^ojects involved a continuation of activities 
begun under previous Cooperative Dpmonstration grants). 
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Problems in Implementing Project Design , Almost all the projects 
encountered some problems in implementing their original design, but 
the intensity of problems varied considerably. One of the most common 
problems among the projects visited was an inability to recruit special 
populations. Many of the projects that had originally planned to focus 
training on disadvantaged persons either shifted their emphasis to 
other clienteles or simply recruited fewer minorities or nontraditional 
students than originally anticipated. The result, overall, was that 
the projects did not attract substantial numbers of minorities or 
nontraditional students by gender. 

The reasons for problems in recruiting varied. One project that 
originally planned to focus on JTPA-referred students found that its 
advisory committee of local businessmen insisted on a level of training 
that demanded academic skills few JTPA-referred students possessed. As 
a result, this project shifted its focus to training current employees 
of companies in the area. Ironically, it found that many of these 
students also lacked sufficient math skills to do the work, and 
instructors ended up teaching basic skills to the employee population. 

Several projects made efforts to recruit black and Hispanic 
students but met with little success. One project attributed its lack 
of success to its inability to provide a technician credential to 
program completers, making it an unattractive 18-month investment. 
Another project was housed in a facility many miles from the grantee's 
main campus, in an area that had no public transportation. A project 
that depended on obtaining training from schools that were accustomed 
to payment from JTPA found that the schools were unwilling to take 
risky students--people who might not finish the training-^because they 
were used to the JTPA payment-for-performance approach. 

The results of these problems can be seen in the survey findings 
on project participants. Males were the largest number of training 
recipients (see Table III-8). For the 79 percent of participants for 
whom gender data was provided, 67 percent were male and 33 percent were 
female. Further, males were concentrated in the projects providing 
training in manufacturing, while females were concentrated in the 
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projects providing training in business services and health care. Five 
of the six projects providing training in health care and three of the 
five reporting training in business services reported a female 
population of 50 percent or greater, compared with only two of the 16 
projects offering training in manufacturing and one of the four 
projects offering training in transportation (see Table III-9). 
Although most projects (19) indicated that they made special efforts to 
attract women, it would not appear that the projects provided a testing 
ground for nontraditional training by sex. 

For the 21 projects that reported the race of most or all 
participants, the vast majority of trainees were white. In these 
projects, whites constituted approximately 83 percent of participants, 
blacks were 11 percent, Hispanics were five percent, Asians were one 
percent, and American Indians/Alaskan Natives were less than one 
percent* Overall, then, whites constituted a greater percentage of 
participants in these projects than they were in the population. Fewer 
than two percent of participants were of limited English proficiency. 
Although 19 grantees said that they made special efforts to recruit 
minorities, among the 21 projects that supplied information, the 
projects appear to have done little to demonstrate new opportunities 
for minorities. However, the seven projects with missing data on race 
include about half the participants. Even if minorities were well 
represented among their participants, they would not have received much 
training because these projects also reported the fewest training hours 
per person* 

The majority of trainees were employed full-time during training. 
Of the 87 percent of participants for whom employment status was 
reported, 85 percent were employed full-time during the period of 
training. Further, most of these persons were employed by the private 
sector partners that cooperated in the projects. Of the 80 percent of 
full-time employees for whom the employer was known, 92 percent were 
employed by a partner company. The total percentage employed by 
partners was probably somewhat less than this figure, of course, 
because employment by the partner was more likely to be known to the 
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grantee. Nonetheless, it is possible to conclude that the main 
training recipients were full-time employees of partner organizations. 

Projects reported some problems due to the logistics of getting 
and maintaining plant and materials. For some projects, projected 
facilities and equipment contributions did not materialize. One 
project failed to gain the funds to build a clean room that was 
critical to its training plan. Another did not obtain the large 
training facility it anticipated in its grant application. One project 
found that loaned equipment was expensive to transport and maintain, 
presenting an unanticipated cost to the project. 

Other projects simply could not execute the designs they 
originally proposed and had to make changes or curtail their offerings. 
One project planned to develop a specialized home automation system, 
but the developers of the system did not cooperate in curriculum 
development. As a result, the project staff had to alter its plan and 
develop a generic home automation installer curriculum instead. 
Another project found that it was much harder than anticipated to 
"sell" businesses on the need for company-wide CIM reforms, and instead 
offered short-term courses in specific computer-assisted applications. 
The three projec^*"^ that planned to develop interactive videodiscs 
underestimated considerably the amount of time necessary for product 
development. None of the three had developed comparable software in 
the past. The project for high school students found it could not 
train students as groups during their internships, and asked students 
to maintain journals instead. The staff also found that internships 
planned to occur dunng the school year had to compete with many other 
out-of-school activities. Most of these problems were resolved, but 
for some projects the changes affected their ability to complete work 
within the 18-month grant time frame. 

The Economic Downturn . Finally, some projects faced problems 
because of the economic downturn, although problems appear confined to 
a small subset of the projects visited. Most commonly, anticipated 
partner organizations were unable to participate or participated at 
severely reduced levels. Further, partners were unable to provide jobs 
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for participants and completers. In one project, the projected partner 
laid off a third of its workers, including the person slated to be the 
main project liaison and an instructor in the program. In several 
projects, poor economic conditions led to low rates of student job 
placement. Nonetheless, the poor economy does not appear to have 
affected the majority of projects--especially those in industries such 
as health care where the conditions may not have been as adverse. 

These site visit findings were borne out by the survey results. 
About half the grantees (14) indicated that economic conditions in 
their communities had deteriorated since they applied for grants (mid- 
1990), but fewer indicated that the changes had affected implementation 
of the grant. When asked whether the decline had affected grant 
implementation, four said that the chan^^es had resulted in fewer jobs 
for students and four indicated that it had resulted in problems for 
the institution. 

Grantees were slightly less likely to indicate that a decline in 
economic conditions in the specific industry or occupation in which 
training was provided had affected project operations. Over half the 
respondents (16) indicated a deteriorating economic condition in the 
specific field(s) of training, but only half of those indicating a 
negative economic change also indicated that it affected project 
operations. Six of these cited poorer job prospects for students as an 
outcome, while three indicated a negative effect on the institution. 
Two grantees indicated that negative changes in the industry created a 
positive opportunity for the project by increasing training needs, and 
one grantee indicated that better economic conditions in the industry 
led to fewer training opportunities at the institution. These last 
responses may reflect a common finding that bad economic times, while 
creating poor job prospects, can sometimes translate into increased 
enrollments in training. Not only do the unemployed seek training, but 
persons who are concerned about losing their jobs may also enroll. 

Deteriorating economic conditions appear to have been most common 
among the projects that provided training in manufacturing (see Table 
III-IO). Seventy-one percent of the projects that provided training in 
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Table III-IO 

MAIN INDUSTRIES AND LIKELIHOOD OF ECONOMIC CHANGE OVER PAST TWO YEARS 

(NUMBER OF PROJECTS REPORTING) 













Industry+ 


Number of 
Projects* 


Improved 


Deteriorated 


Remained 
the Same 


Construction 


4 


0 


1 


3 


Manufacturing 


17 


2 


12 


3 


Transportation 


4 


0 


2 


2 


Business Services 


5 


1 


2 


2 


Health Care 


7 


1 


2 


4 


Education 




0 


2 


2 


Multiple Industries 


7 


1 


4 


2 



+ Industries w/fewer than four responses omitted from analysis, 
* Adds to more than 27 because projects could select more than one 
industry. 
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manufacturing reported that economic conditions in the industry of 
training had deteriorated since the grant application. Among those 
grantees providing training in health care, only 29 percent reported 
deterioration, while among those providing business services, 40 
percent indicated economic deterioration in the industry. As trainees 
in projects offering manufacturing training were the most likely to be 
full-time workers during the training, however, the economic 
deterioration may not have translated into immediate job effects. 

5. Exportabilitv of Project Activities and Products 

Because this was a demonstration program, the study team sought to 
determine what project features had applicability to, or could provide 
useful models for, other educational or employment training programs. 
What elements of these projects were worth replicating widely, as had 
been anticipated by the regulations governing this program? 

Survey Findings . The projects were quite divided in choosing 
features of their projects that could be used or applied by others. 
Respondents were provided with a list of possible project features and 
asked to rank up to three features of their projects they considered 
applicable or useful for others. The most commonly selected first 
choice feature was customized training for a particular employer or 
group of employers (six respondents). The second most commonly 
selected feature was a new or improved curriculum (selected by five 
respondents). "New or improved kind of training," "established or 
strengthened public-private partnerships" and "applied high technology 
equipment to the delivery of training" received four responses each. 
Three projects chose "expanded access to training for disadvantaged or 
underrepresented groups." Only one selected "model of school-to-work 
transition," which wa^ a demonstration objective specifically 
identified in the Federal grants announcement. 

This wide range of responses can be interpreted several ways. 
Given that the program was intended to be a demonstration of public- 
private partnership for training in high-tech fields, the lack of 
consensus on exportable features is surprising. One might have 
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expected the partnership or training choices to account for a large 
number of first choices, but they account for only eight of 27. The 
selection of customized training as a first choice is notable, because, 
by its nature, customized training is rarely applicable across 
industries or educational institutions. 

If all three choices are added together, however, establishing or 
strengthening public-private partnerships appeared to be the feature 
most commonly selected as having wider applicability. Twenty projects 
considered partnership among their top three exportable features. 
Conversely, few projects that did not select customized training as a 
first choice selected it second or third--making it one of the lowest 
ranked choices overall. Beyond partnerships, however, no exportable 
feature was identified by more than half the projects, even with first 
through third choices added together. 

The lack of consensus on what is being "demonstrated" is 
troubling. While it is not possible to reach conclusions from the 
responses to one question, it seems that grantees do not appear to 
share a common view of what they used Federal support to "try out." 
And although all projects were expected to "try out" public-private 
cooperation for high-tech training, only 20 projects considered their 
new or improved public-private partnerships, and only 13 projects 
considered their training, as applicable to or providing a useful model 
for others. 

Site Visit Findings . The site visit teams tried to make an 
assessment of what could be replicated from the various projects. They 
considered two criteria. First, they looked for some evidence that the 
project succeeded in accomplishing its goals, i.e., did it do something 
worth considering for replication elsewhere? If the project met the 
first criterion, it then needed a mechanism for facilitating 
repl ication--a product or other means for guiding other<; to implement 
the same set of activities. 

For some projects, particularly those concerned with curriculum or 
product development, evidence of project success and repl icabil ity 
could be easily determined. These projects were, essentially, "tests" 
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of the efficacy of the curricula or other '^taterials/products (such as 
interactive videodiscs) the projects developed. The grants enabled the 
cu^^'icula or other products to be developed and installed in a pilot 
site or sites. The experiences of the users in those sites were 
observed and/or their comments solicited in a relatively systematic 
manner. Sometimes changes were made and the product was re-piloted. 
At some point, the developers were either satisfied or unsatisfied with 
the quality of the product. 

There were only a handful of projects that proceeded in this 
manner, however. They included the projects that developed curricula 
for home automation installers, semiconductor manufacturing employees, 
and metal fabrication. They also included the high school occupational 
clusters training and the two projects that developed interactive 
videodiscs that can be used in instructional settings. All of these 
products have been pilot tested through the projects in one or more 
sites and were nearing or at completion at the end of the grant period. 
Seeing how well they can be used by persons not receiving direct 
assistance from the developers could be the next step in determining 
their exportability. 

Beyond these projects, it less clear what project activities or 
partnerships were exportable or applicable elsewhere. First, some of 
the closest partnerships—such as the two steel industry projects- 
provided a training experience and direct instruction that may not have 
wider applicability. At best, they present a general lesson for 
others—that colleges can perform valuable assistance in helping 
manufacturers convert to the use of high-tech equipment. Working 
closely with managers, instructors can design and deliver a mix of 
classroom instruction and OJT to employees. Unfortunately, most of the 
projects were not accompanied by the kinds of documentation or process 
evaluation that would explain how such partnerships should or should 
not proceed to be most effective. 

Second, beyond the curriculum or product development projects, the 
lack of systematic evaluations addressing project effectiveness makes 
it difficult to draw conclusions about project success (the 
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evaluation's first criterion for replication). Most projects did 
engage "third-party" evaluators, but their evaluations either focused 
exclusively on process (i.e., they showed that projects met their 
obligations with respect to number of students, amounts of training, 
etc.) or lacked sufficient rigor with respect to assessing outcomes to 
reach conclusions. For example, a number of projects showed low rates 
of student placement. Because the projects did not conduct evaluations 
with comparison or control groups, it was impossible to know the extent 
to which the lack of placements was due to the poor economy or to 
inadequate project activities or procedures. It should also be noted 
that several projects did not have completed "third party" evaluations 
at the time of the site visit (although the vast majority of projects 
were completed or in their last month). Despite the "demonstration" 
intent of the Cooperative Demonstration Program, evaluation was treated 
in a pro-forma by many projects; it was largely an afterthought to 
service del i very , 

A few projects did develop guides for implementing their 
procedures. For example, the project that assessed students and 
referred them for training to local schools and colleges produced a 
manual on its experience. Several projects provided the site visit 
teams with examples of curricula developed by staff supported, in part, 
with project funds. In the absence of any evaluations of the impact or 
effectiveness of these projects, however, it is hard to know whether to 
encourage others to use the same approaches, or even whether it would 
be worth the effort to test these interventions further. 
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D. Key. Findings 



1 . Pro.iect Training 

Overall, projects focused on providing relatively short-term 
training. The typical participant received fewer than 100 hours of 
training. The projects included in the site visits can be grouped into 
a few basic categories: 



■ Short-term, skill-specific instruction for 
current employees of companies, or for 
persons already familiar with the field of 
training. This approach was most common 
among manufacturing projects, but it could 
also be seen in the refresher nursing course 
and in business projects teaching specific 
software applications, as well as home 
automation installation. Sometimes 
instruction was customized, tailored to the 
needs of specific employers. 

■ Medium-term, entry-level technician training. 
These were programs in which training ranged 
from a few months to a year, often aimed at 
helping unemployed or otherwise disadvantaged 
persons find jobs in a field. These programs 
cut across the industries but were quite 
common in business. The three projects run 
by JTPA-supported agencies fell within this 
category, one of which provided training in 
several fields. Some provided certificates. 

■ Longer-term programs aimed at two-year 
certification or degrees (associate degree, 
license). Some of these projects were aimed 
at institutional capacity-building and 
regular students, e.g., creating new or 
revising existing offerings within depart- 
ments of industrial technology or business/ 
computers in community colleges. One was an 
apprenticeship training project, and one an 
occupational program for high school 
students. 
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2. Partnerships 

Among projects included in site visits, intensities of partner- 
ships varied from two projects where single employers played critical 
roles in service development and delivery, to several projects in which 
employers did little more than attend meetings of an advisory 
committee. According to survey results, most partnerships fit one of 
two models: 

I The partner was a customer of the project, 
e,g, the project provided customized training 
to the partners' employees; or 

■ The partner shared actively in the delivery 
of instruction and services. 

Projects with multiple partners were considerably more likely to report 
that partners were customers, while projects with few partners used 
partners in more ways, including active sharing of instruction. 

3. High Technology 

Initial concerns that projects might not have a high-tech focus do 
not appear to be have been reflected in practice. This is true 
primarily because almost all fields have some high-tech elements. If 
learning to operate high-tech equipment is included as a training 
option, requiring that the project be high-tech in focus excludes 
almost no training effort. 

4. Operation and Integration of Projects within Institutions 

FY1989 projects reported fewer start up problems than did FY1988 
projects. The 18-month time frame continued to present problems in 
completing project activities however. Whether a project started 
providing services shortly after award made little difference in 
whether it continued after the grant ended. The site visit teams 
identified several problems that cut across projects. They included: 
an inability to recruit special populations as planned, problems in 
getting and maintaining plant and materials, poorly developed designs 
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that could not be executed, and underestimating the time necessary for 
product development. Aside from the recruitment difficulties, most 
problems were resolved but had an impact on those projects' ability to 
complete work within the 18-month grant time frame. The poor economy 
did not affect the majority of projects— especially projects in 
industries such as health care v;here the conditions may not have been 
as bad. 

5. Exportabil itv of Pro.iect Activities and Products . 

Survey results suggested that projects lacked consensus on what it 
was that the Cooperative Demonstration Program was "demonstrating." 
Only 20 projects considered their public-partnerships, and only 13 
projects considered their training, as applicable to, or providing a 
useful model, for others. Site visits revealed that most projects 
simply did not conduct the kinds of evaluations that would allow 
possible replicators to determine whether the project was successful 
for participants. Projects that developed and tested discrete 
curricula or products were more likely to have some evidence of 
effectiveness and a product to disseminate. 
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PROJECT COSTS AND BENEFITS 



IV-1 

IV. PROJECT COSTS AND BENEFITS 



Were project costs reasonable relative to the projected or actual 
outcomes of the project? Costs were defined as the Cooperative 
Demonstration grant plus non-Federal cash match and in-kind 
contributions of the grantee. The outcomes of the project are defined 
as numbers of students trained (if training is the project focus) 
and/or the number of course hours developed (if curriculum development 
the project focus). "Reasonableness" is judged by comparing costs and 
benefits among projects, not against some absolute standard. These 
analyses quantify major project outcomes and, where possible, the total 
costs of achieving them. No attempt was made to assign a monetary 
value to the benefits resulting from the project outcomes (e.g., the 
dollar value of learning a new skill). Also, the analyses excluded 
consideration of the costs in time and effort to students participating 
in the projects. 

This section is divided into four subsections. Subsection A 
identifies major cost/benefit issues and the operational definition of 
those issues. Subsection B defines and enumerates project costs, and 
subsection C defines and enumerates major project benefits. Subsection 
D compares costs with benefits in accordance with the major issues 
raised in Subsection A, 
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A, Major Issues 



Project costs and outcomes are aggregated to estimate overall 
costs and benefits. These are used to compute four measures: 

■ Treatment costs : total project costs minus 
planning and development costs; 

■ Project intensity : total treatment hours 
divided by the number of successful comple- 
tions; 

■ Average unit cost of delivered services : 
total treatment costs divided by total units 
of service; and 

■ Service cost per unit of outcome : total 
treatment costs divided by the number of 
successful completions . 

Operational definitions of treatment hours, successful completions, and 
units of service produced vary according to the type of treatment 
(e.g., training versus curriculum development)* For projects focusing 
on student training, the number of treatment hours are the number of 
classroom contact hours; the successful completions are those students 
finishing the course; and units of service are hours of training per 
student. 

The four measures were analyzed only for the 19 FY1989 projects 
for which data were collected during the site visits (see Section III). 
The site-visit teams collected limited cost and outcome data (e.g., 
project budgets and numbers of students trained) from each project 
through staff and partner interviews and review of budget and 
expenditure reports, project progress reports, and the mail survey of 
all FY1989 projects. Project records showed direct expenditures from 
grant and from non-grant sources. Partner organization members were 
interviewed to determine direct expenditures or in-kind contributions 
to the project. Finally, project staff were interviewed to identify 
other in-kind contributions to the project, such as donated equipment 
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or release time from other organizations. Although not all in-kind 
contributions could be translated into dollars, the study team made 
rough estimates of the value of the contribution stated in the grant 
application or during staff interviews. 
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B, Project Costs 



Project costs are the financial and nonfinancial resources used 
for project activities* Since Cooperative Demonstration grants 
required grantees to contribute at least 25 percent of total project 
cost, project costs include both the Cooperative Demonstration grant 
amount and grantee match. Project costs often included more than this, 
however. Some projects used existing instructional services paid by 
the grantee as part of their regular operations. Others used staff and 
materials funded from other sources to supplement the activities of the 
Cooperative Demonstration project. These outside sources also should 
be included in the calculations of total project costs. Thus, 

Total Project Costs = Federal grant + non-Federal match + 
outside project resources. 

Unfortunately, few outside sources could be included in the calculation 
because the grantee had not track them. Thus, the analysis is based 
only on the direct services and materials paid for with grant funds. 

1. Cost Categories 

The primary components of project costs used in the analyses were 
those in line-item budgets, including: 

■ Salaries and Wages; 

■ Fringe Benefits; 

■ Travel ; 

■ Equipment; 

■ Supplies; 

■ Contractual Services; 

■ Other Direct Costs; 

■ Total Direct Costs; and 

■ Indirect Costs. 

Salaries and Wages . Primary staffing costs were salary and wages 
paid to staff, including annual salary and hourly wages for all 
employees of the grantee--or the partner organization— who work on the 
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project. Staff included teachers, instructors, administrators, other 
certified personnel, clerical staff, and support staff. 

Fringe Benefits . Employees usually received fringe benefits as 
part of their compensation package, including sick leave, annual leave, 
holidays, health insurance, etc. Fringe benefits are usually 
established by the institution as a percentage of total salaries and 
wages • 

Travel . Any travel costs were listed as a separate line item, 
including airfare, car rental, ground transportation, hotels, meals, 
and tips. 

Equipment . Although the program discouraged FY1988 grantees from 
using grants to purchase equipment and prohibited it in FY1989, some 
projects did purchase new equipment. Other projects used non-Federal 
sources to pay for equipment. 

Suppl ies . Projects required routine office supplies, specialized 
materials, and instructional materials. 

Contractual Services . Some projects hired outside experts or 
temporary personnel, including outside evaluators or specialized 
services provided by other companies through contracts. Rules 
governing the use of consultants (individuals) differ from those 
governing the use of companies (subcontracts), but both involve the 
external acquisition of services and are grouped in one category. 

Other Direct Costs . Direct costs that do not fit into the above 
categories are listed as "other costs" and may include space rental, 
telephone, and postage if they are not paid for indirectly (see 
Indirect Costs below) . 

Total Direct Costs . The sum of all the direct cost categories 
yields the total direct costs of a project. 

Indirect Costs . Indirect costs are charges made by grantee insti- 
tutions for overhead items such as office space, heat, electricity, 
postage, accounting services, and administrative services. Grantees 
usually provided these to all projects and programs. To pay for items 
that are difficult to itemize, the grantee institution charged projects 
an indirect cost, or overhead, rate. The indirect cost was usually 
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based on a percentage ot the total salaries and fringe benefits, but 
may also include other direct costs in the base. Under the FY1988 and 
FY1989 grant regulations, grantees were allowed to charge a maximum 
indirect rate of 8 percent. 

Total Project Costs . Total project costs were the sum of all 
direct costs and all indirect costs associated with the project. 

2. Sources of Funding 

Projects funded activitijs from three major sources: grant funds 
provided by the Federal government through the Cooperative Demonstra- 
tion Program grant award; non~Federal cash or in-kind resources from 
public or private organizations (e.g., the grantee or partner organiza- 
tion) to meet the required 25 percent match; and funds or in-kind 
contributions not identified in the grant application nor reported in 
the project expenditure reports (usually underlying instructional or 
support services provided by the grantee as part of the regular 
educational program). In-kind resources included grantee staff time, 
partner staff time, equipment, facilities, services, materials, and 
information. Grantees did not always identify these resources because 
1) the grantee already had satisfied the 25 percent match, 2) it didn't 
have the resources to track them, 3) it needed them offset any of the 
25 percent matching funds that may be disallowed in a subsequent 
Federal audit or 4) the accounting system could not handle them. 

The total costs of each project and sources of funding are shown 
in Table IV-1. Total project costs ranged from $261,274 at Nebraska 
Department of Labor to $1,355,966 at Clackamas. It should be noted 
that grantees may have overestimated the value of the partner's 
contribution or given vague figures for equipment. 
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Table IV-1 
TOTAL PROJECT COSTS 



PROJECT 


FEDERAL GRANT 
FUNDS 


LOCAL CASH 
MATCH 


LOCAL IN-KIND 
MATCH 


OTHER PROJECT 
RESOURCES 


TOTAL PROJECT 
COSTS 


Bronx Conmunity 
Col lege 


473,549 


0 


460,000 


0 


933,549 


Clackamas 
Comnum' ty 
Col lege 


366,305 


0 


990,661 


0 


1,356,966 


CORD 


417,880 


8,270 


110,662 


117,755 


654,567 


Fox Valley Tech 


437,727 


0 


0 


326,900 


764,627 


Hampden County 
Consort inn 


359,309 


0 


124,800 


0 


484,109 


Institute 


408,318 


0 


30,000 


0 


438,318 


Col lege 


214,924 


77,997 


15,325 


0 


308,296 


College 


178,270 


0 


105 400 


196,571 


480,241 


Indian Hi lis 
College 


239,089 


98,434 


48,736 


0 


386,259 


LTV Steel Co. 


252,821 


839,319 


0 


0 


1,092,140 


Luzerne 
Comnunity 
Col lege 


234 , 1 74 


0 


180,218 


0 


414,392 


Nebraska Labor 
Department 


171,874 


0 


89,400 


0 


261 ,274 


North Clackamas 
Schools 


257,274 


23,850 


138,930 


185,000 


605,054 


PAVE 


383,385 


5,442 


180,750 


0 


569,577 


Southwestern 
College District 


278,379 


238,213 


138,375 


0 


654,967 


State Center 
College District 


399,000 


0 


275,945 


0 


674,945 


Waubonsee 
Conmuni ty 
Col lege 


243,328 


102,201 


0 


0 


345,529 


West Virginia 
Department of 
Education 


377,086 


11,884 


196,686 


0 


585,656 


West Virginia 
Northern College 


471,808 


0 


161,439 


200,986 


834,233 



* tatal project costs = Federal grant ♦ local cash match + local in-kind match 
+ other project resources. 
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C, Project Benefits 



Project benefits are improvements in the ability and/or employment 
status of students or in the quality of vocational education. Some 
outcomes are quantifiable (e.g., the number of students successfully 
completing training), while others are nonquantifiable (e.g., a new way 
to identify skills needed by local employees). Nonquantifiable 
outcomes cannot easily be compared against project costs. 
Consequently, the following cost/benefit analyses use only quantifiable 
outcomes. 

Quantifiable outcomes vary with the type of activity and the focus 
of the project. For example, the benefits of curriculun development 
are new teaching modules or materials. Benefits of disst„iination 
projects include "how-to-do-it" descriptions of the project, 
publications and presentations, or greater awareness by other 
vocational education institutions. Similarly, benefits of training 
include the number of students with documented improvement in skill 
levels. 

The Statement of Work (SOW) for this contract identified three 
quantifiable outcomes to be measured: 1) the total number of service 
hours, 2) the number of successful participants, and 3) the number of 
person hours of services received per participant. For projects aimed 
at improving vocational education, primary outcome measures were: 1) 
the total hours of student training provided, 2) the number of students 
completing the training, and 3) hours of training received by each 
student. 

The grants funded other activities that provided services as well. 
Their value was measured through the three general measures defined 
above. The following are the quantif icable outcomes, by type of 
activity, that were measured for each project: 
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Activity 



Outcome 



Student assessment 



number of students assessed 



Training students 



number of students completing 
training 



Staff development 



number of staff trained 



Curriculum development 



number of course hours developed 



Skills identification 



number of industry skills 
identified 



Dissemination/diffusion 



number of other end users 
adopting product 



Partnerships/networking 



dollar value of partner(s) 
contributions 



The major outcomes from each type of activity are summarized in Table 
IV-2. These measures do not capture aU possible outcomes. Many 
important benefits could not be measured until after the end of the 18- 
month grant period. For example, while the number of students hired is 
one measure of the quality of training, no data on employment were 
available if the students had to undergo further training for a job or 
if they were already employed. The demonstrated mastery of technical 
skills is the ultimate measure of the effectiveness (along with job 
placement) of a new curriculum, but most projects did not conduct 
pretests because no such tests existed. Thus, available outcome 
measures often told what was done, but not how well it was done. 
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D. Cost/Benefit Analyses 



Ideally, the project team would determine cost effectiveness by 
matching costs to specific activities and outcomes. However, the team 
could not collect cost data for each outcome separately for two 
reasons. First, grantees did not code expenditures by activity because 
their accounting system did not operate at that level of detail, i.e., 
expenditures were aggregated only according to budget line items. 
Second, several activities usually contributed to the same outcome- 
projects implemented activities or one activity was used to support 
more than one objective. 

The team analyzed the three activities for which cost data were 
available through the grantee's accounting system or the final contract 
budget: planning and administration, student training, and curriculum 
development. Planning and administration costs include wages of the 
project director and clerical staff, associated fringe benefits, other 
direct costs associated with administration, and indirect costs. 
Student training costs include the wages of instructors and other 
specialists, associated fringe benefits, other direct costs (e.g., 
textbooks, supplies, travel, and stipends), and indirect costs. 
Curriculum development costs include the wages of instructors and 
curriculum development specialists, associated fringe benefits, other 
direct costs (e.g., training workshops, travel, and printing), and 
indirect costs. 

1 . Treatment Costs 

The first major analysis separated planning costs from the costs 
of providing the service. Treatment costs equal total costs less 
planning and administrative costs: 

Treatment Costs = (total project costs) - (project director 
wages + clerical wages + fringe benefits + 
other direct costs + related indirect costs) 
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Treatment costs and planning/administration costs for each project are 
displayed in Table IV-3. The proportion of total project resources 
devoted to planning and administration ranged from 9.5 percent at LTV 
Steel to 48.0 percent at PAVE. 

2. Project Intensity 

The intensity of the project (i.e., the number of successful 
outcomes relative to the effort expended to accomplish those outcomes) 
is defined as: 

Project Intensity = total treatment hours 

number of successful completions 

The effort to train students was the sum of the hours students spent in 
training in all courses. The number of successful outcomes was the sum 
of the students completing training. In projects where the treatment 
was staff training, the number of successful outcomes was the number of 
teachers completing training. Project intensity for each activity for 
which data were available--training students and training staff--are 
displayed for each project in Table IV-4. The amount of training per 
student completed, project intensity, ranged from 16.92 hours at 
Southwestern Community College to 7,000 hours at LTV Steel. Staff 
training, offered formally at only three sites, ranged from 20 hours 
per teacher to 53.5 hours per teacher. 

3. Average Unit Cost of Services 

The average cost per unit of service is the total cost of the 
service divided by the number of units provided: 

Average Unit Cost = total treatment cost 

of Service total units of service provided 

For example, total service costs for providing student training was the 
sum of the project costs for staff, materials, overhead, etc. for all 
the courses. Total service costs for curriculum development was 
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the sum of project costs for staff, testing, reproduction, overhead, 
etc,, for all the courses produced. Total units of service provided 
was the sum of all students entering training or the number of new 
courses. 

Average unit costs for student training and curriculum development 
for each project are displayed in Table IV-5. The average unit cost 
for student training ranged from $.45 at PAVE to $40.47 at State 
Center. 

Comparing the average cost-per-hour-of-training across projects 
may create an unfair comparison because of variations in the intensity 
of the training and the number of students being trained. These 
differences affect the comparison of costs for curriculum development. 
The average unit cost for curriculum development (cost per course hour) 
ranged from $1.69 at the West Virginia Scate Department of Education to 
$24,179.90 at Howard Community College. However, comparing the average 
cost~per-unit-hour for curriculum development also may be misleading. 
Howard^s course was an interactive videodisc, which had a much higher 
initial development cost due to the technology used. 

To provide a more accurate comparison, the analysis should 
compute the average cost per hour of training per student trained . 



Table IV-6 shows the per-hour per-student costs for the projects. 
The per-hour per-student costs range from $.001 at West Virginia 
Northern to $9.24 at Luzerne Community College. The costs for the 
remainder of the projects tended to concentrate under $1.00 an hour. 

4. Service Cost Per Unit of Outcome 

The service cost per unit of outcome is defined as total service 
costs divided by the number of completions: 



Average Unit Cost 
of Service per 
Unit of Outcome 



total units of service provided 



total service costs 



number of completers 



Service Cost = total service costs 

per Unit of Outcome number of successful completions. 



IV-17 



> 



a. Lu 
>^ Of 

o u ac 



25 



2S 



W • »- 

U Z 

(/>*<- LU 

O oe Z 



O 

o 
o 



o 



;8i 



8: 



:8 



o 



V> -J 

or =3 

3u o 
» Ui 

= 2a 

_J ZJ -J 
< U UJ 

»— > 

O U. LU 

»— o o 



o 

00 

o 



o 
o 



o 



o 



o 



O ' 



at 

(/> LU 



t~ oe < 

O O oc 



o 



o 

o 



3 



00 



O 
O 



B S o 

^ Z 

<«~ 

O u. oe 
>- O »~ 



:8 



8: 



R 



CD — ' 
-» O 

u u 



c 
o 
u 



o 
u 



o 
u 



s. 

o 



C/1 4^ 
3 • 



O 



c 



o 
u 



(A — ' 

a o 

3 U 



«3 , 



o 



on 
O 



> 



ERIC 



IV-18 



! 



o 



o 

u. 
o 

h- 
»i— « 
2Z 

r5 

UJ 

a. 

00 

o 
o 



C5 



u. oc 
O Ul 



11 



UJ u. z 
a. O ^ 

X 

»— oc 
</> 2 < 
O S oc 



C/J UJ 

O Q <D 

O 5 Z 

^ (/) z 

O O oc 



8 S o 

a= ^ at 

~j ac 
< to 

»- < 

O u. oc 

o • 



8: 



o 
o 
o 



3 



C 
L. 
41 

O U 

to O 



i8 



8: 



1 = 



00 

rj 

00 



CsJ 
00 



a 
o 



c o 



o 
o 

CO 





•TO 












•TO 












> 






o 






s. 






Q 








t/) 








O 


c 


0) 


u 










0) 








C 




'a. 


O) 


0) 








o 






u 












^- 


u 


O 


o 










(/) 


w 






a> 


<0 




JO 




c 


E 


O 


3 








C 










<« 












O 

















00 






O 






U 




£ 






O 




o. u 


•r— 






C 








u 


O 


O) 


> ^- 




O 


ro 


a> 




s. 


(/) 










> «4- 


c 


<9 


o 


3 



ERIC 



IV-19 

The number of successful completions is defined as the sum of the 
students completing the training and the number of courses successfully 
developed. The unit service costs for training students and curriculum 
development are shown for each project in Table IV-?. Service costs 
for student training ranged from $190 at West Virginia Northern to 
$62,971 at Bronx Community College, For curriculum development, the 
cost per course ranged from $233 at Indian Hills to $241,799 at 
Hampden. The relatively high cost at Hampden County again was due to 
the high cost of the interactive videodisc. 

In summary, the answer to the question "are project costs 
reasonable in relation to project outcomes?" appears to be yes for all 
projects. The per-unit and per-outcome costs for all the other 
projects tended to cluster in the same area even though total costs and 
project intensity varied substantially. 
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V. ISSUES IN FEDERAL PROGRAM MANAGFMFNT 



Although the Cooperative Demonstration Program (High Technology) 
has ended, there are important lessons to be applied to future demon- 
strations sponsored by OVAE and the Department. The experiences from 
the 27 FY1988 and FY1989 grantees visited by the study team suggest at 
least two groups of issues in Federal management of cooperative 
demonstrations: 

A. Specifica tion of the purpose of and roles in the 
demonstration 

" Policy-implementing versus policy-formulating 
demonstrations; 

■ Use of evaluators; 

■ Partner commitments; 

■ OVAE support of grant changes; 

■ Utilization of all available resources; 

^- Administration of t he application and grant award processes 

■ Instructions to applicants; 

■ Timing of the grant award; and 

■ Length of the grant; 

Each of these issues is discussed below. Based upon the results of 
this evaluation, the study team recommends alternatives for the design 
and administration of future demonstrations. 
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A. Clarifying the Demonstration Program 



The Purpose of the Cooperative Demonstrations 

Although titled "Cooperative Demonstration Programs," the 
authorizing legislation did not clearly define the term demonstration 
or the intent of the legislation. Instead, the legislation referred to 

programs and projects which support model 
programs. • .examples of successful cooperation 
between the private sector and public agencies in 
vocational education. . .programs to overcome 
national skill shortages. . .and activities such as 
institutional and on-the-job training, supportive 
services, and other such necessary assistance... 
(Section 411). 



The legislation did not specify whether projects were to demonstrate 
that an intervention could be successful if it hasn't been tried 
before, could be improved in its original site if already in operation, 
could be successful in a new site if already implemented elsewhere, or 
some combination of these intentions. However, section 411(c)(2) did 
require that all funded programs be "capable of wide replication by 
service providers," suggestii.j chat Congress intended the programs to 
demonstrate approaches that had applicability beyonc just one grantee. 

The program regulations also were unclear about what was to be 
demonstrated. The regulations specified that projects. 



must be designed to demonstrate ways in which 
vocational education and the private sector of the 
economy can work together effectively to assist 
vocational education students to attain the 
advanced level of skills needed to make the 
transition from school to productive employment. 
[CFR 112.10(a)(2)(ii)] 



This broad definition allowed a wide variety of projects to be 
funded. Although all projects satisfied the general conditions of the 
regulations, it is questionable whether all projects met the intent of 
the regulations and the Perkins Act. For example, is a grant to a 
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single company to retrain current employees in training programs the 
company would have done anyway a legitimate purpose under the program? 
OVAE should clearly state whether the demonstration is to field test 
previously identified public/private partnerships or to develop new 
models of such cooperation. 

A key characteristic of demonstrations is that they involve "...an 
innovation operated at or near full-scale in a realistic environment" 
(Glennan, et al . , 1978). Further, demonstrations can be divided into 
two types: policy-implementing or policy-formulating. Policy-imple- 
menting demonstrations take research ideas and put them into practice 
in the real world, while policy-formulating demonstrations generate new 
ideas and approaches based on actual field experiences (Yin and Sabol, 
1991). The knowledge from policy-implementing demonstrations is 
focused on the experiences and outcomes from implementing ideas 
previously found effective under controlled research conditions. The 
knowledge from policy-formulating demonstrations is focused primarily 
on identifying ideas which can be further tested under research condi- 
tions. In both cases, demonstrations cannot be expected to establish 
cause-and-effect relationships because the real-life settings involve 
confounding events (Ginsburg, 1989). 

Other Federal agencies have been more specific in their stated 
intent for funded demonstrations. For example, the Office for 
Substance Abuse Prevention, U.S. Department of Health and Human 
Services, stated that the purpose of one demonstration program was to: 

1. Test the feasibility of implementing 
previously untested innovative prevention 
strategies that hold great promise for 
expanding our repertoire of strategies and 
interventions in the prevention of alcohol 
and other drug abuse among high-risk youth 
populations; or 

2. Assess the program effectiveness, repli- 
cability, and general izabil ity of 
knowledge-based, established strategies for 
the prevention of alcohol and other drug 
use among high risk youth populations, 

l&l 
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including those derive from previous OSAP 
experience but not systematically 
evaluated- 



Applicants were required to: 



Indicate whether this proposal is to (1) test 
the feasibility of implementing innovative 
prevention strategies; or (2) assess the 
effectiveness, replicabilitv. and gener- 
al izabilitv of knowledge-based, established 
strategies for the prevention of and/or early 
intervention against alcohol and other drug 
use among high-risk youth . If this applica- 
tion is proposing an innovative strategy, it 
is incumbent on the applicant to explain from 
the literature review what makes this ap- 
proach innovative. If this application is 
designed to assess effectiveness, gener- 
alizability, and replicabil ity of established 
strategies discuss what is known about the 
established strategies and how this applica- 
tion address general izabil ity and replic- 
abil ity of those strategies. 



Partner Commitments 

Projects funded by the Cooperative Demonstration Program were to 
demonstrate how public agencies in vocational education and the private 
sector could effectively and successfully work together. The regula- 
tions do not suggest what role the private sector should play, nor is 
the program designed to produce information on what roles were most 
effective for the private sector. This meant that each project was 
left to deal with developing the private sector role as it saw fit. 
The private sector was to be involved in the planning as well as the 
operation of the project (five points was awarded in the application 
for private-sector involvement). The role of the private sector was 
not well defined in the program regulations, and, as discussed in 
Section III, varied widely across projects. Among the FY1989 projects, 
the private sector was: the grantee, the referral source of students, 
and employer of program graduates, a source of internships, a member of 
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the advisory group, a supplier of equipment, or a source of 
instructors. 

The lack of specificity in the regulations and application 
requirements regarding private-sector involvement appears to have been 
problematic for several projects. Because the regulations were vague 
about what was acceptable private-sector involvement, applicants 
similarly were vague in their descriptions of how private companies 
would be involved in their projects. The vagueness in the applications 
sometimes was an indication of lack of planning on the part of the 
grantee. Other times it was the result of the grantee not being able 
to get the private partners to commit to hiring students or to 
activities other than advising the project. 

Grantees reported problems in implementing their projects because 
of changes or problems in the partner's role. Some of these problems 
might have been avoided through additional planning in the application 
process. One problem was that the role of the partner was left very 
general during the application, and the partner was reluctant to be 
specific after the grant started. Some grantees overstated what the 
partner was to do. For others, a change of conditions made it 
impossible for the partner to fulfill its planned role. 

The partnerships described in the applications appeared to be much 
stronger and more formal than those actually observed during the site 
visit. The statement that the applicant "has been working with Ajax 
Corp," for several years can mean many things. It can mean that Ajax 
has supplied instructors or equipment or it can mean that Aiax has 
hired graduates of the institution. 

Partners were asked to submit letters of support for the applica- 
tion. However, the partner might never have seen what the applicant 
stated in the application and might not have been willing to provide 
the kind of support indicated. On the other hand the partner may have 
made substantial oral promises that were not committed in writing. The 
applicant assumed that it had a formal commitment based on the oral 
statement, but the partner did not feel it was a firm commitment. 
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The applicant should be required to have a signed document from 
each partner stating that the partner has read the proposal and is 
willing to commit to the support stated therein. After the award, and 
during the planning stage, the grantee should enter a formal agreement 
with the partner detailing precise responsibilities. Otherwise the 
project can be left without adequate support. 

Partner support has to be clarified. Does it mean a guarantee to 
hire? to refer students? to provide equipment? to provide technical 
assistance etc.? Many grantees considered as partners the businesses 
that simply hired its students, even though the businesses had little 
involvement in the training process. 

Use of Third-party Evaluators 

Grantees* use of third-party evaluators was limited to verifying 
project activities. Although all projects had third-party evaluators, 
they were seldom used to measure the effectiveness of project 
activities or help determine the relative worth of the components of 
the project. Instead, grantees used the outside evaluators to confirm 
that activities had taken place. Most projects set aside $3,000 to 
$7,000 to hire a single outside person to conduct the evaluation. The 
evaluator observed several classes, talked to several students and 
faculty, and produced a report of a few pages. In a few instances the 
evaluator was used to assess the curriculum materials adopted by the 
training rather than the project itself. 

The relationship between the evaluator and the project was not 
always unbiased. In one instance, the third-party evaluator was the 
same organization that had provided assistance in writing the grant 
application. Afterward, the evaluator provided technical assistance as 
well as evaluation services. The two roles had the potential of being 
confl icting. 

Utilizing All Available Resources 

Many of the projects engaged in curricula development either as a 
primary objective of the grant or in response to requests for specific 
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training from a partner organization. In almost every case, the 
project developed the new curriculum without drawing from existing 
curriculum materials from other institutions, from ERIC, or from the 
OVAE regional curriculum coordinators. When asked why the project did 
not use-or at least review-the curriculum materials available 
elsewhere, project directors responded either that their training was 
unique and that materials developed elsewhere would probably not apply 
or that there were no materials elsewhere. A few project directors 
reported that they checked with other grantees, but only one reported 
conducting a systematic search for materials. 

The failure to conduct a systematic search of other institutions 
and the lack of communication among the grantees resulted in some 
duplication of effort in curriculum development. Although some pro- 
jects had unique elements, the program requirements for demonstrating 
successful cooperation in vocational training and for repl icabi 1 ity 
suggest that the project already should be grounded in the existing 
literature and curriculum for the subject in their training area. With 
the scarcity of Federal and non-Federal resources, it is important to 
minimize any duplication of effort. 

To reduce or eliminate duplication of effort in curriculum design 
OVAE may want to require applicants to show evidence of a systematic 
search for curricula as part of their proposal development. If OVAE 
thinks such a systematic search is too burdensome during the applica- 
tion stage, then the search should be required as part of the initial 
six-month planning grant described earlier. Even if a search is 
conducted during application preparation, an updated search should be 
made during the planning grant stage to identify any new curriculum 
packages published during the previous 12 months. 

PVAE Support of Chan ges bv firantPPc 

Several projects encountered operational problems during the grant 
period. Problems included a deteriorating or shifting labor market 
withdrawal of a key partner, or discovery that an activity was more' 
difficult or expensive than originally estimated (e.g., producing a 
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videodisc). In reviewing the implementation history of the project, 
the study tear thought the problems were probably severe enough to 
warrant a change in project design or objectives, although such changes 
were made in only one case. During the site visits, the team asked 
project directors and jtaff why they didn*t request a change in the 
grant's scope to keep up with changing conditions. Almost all said 
that they thought they had to fulfill the terms of the grant 
application and that any requests for changes would be considered a 
sign of failure or simply not approved. This misperception on the part 
of project staff resulted in the project continuing on an 
inappropriate~~and ultimately unsuccessful-~path. 

There were several reasons why the OVAE program managers might 
have been unaware of all the implementation problems encountered by 
grantees. First, not all projects submitted the required quarterly 
progress reports, and those that did often did not include a complete 
discussion of any problems being encountered. Second, even during the 
periodic telephone calls by program managers to the project, project 
directors might have been reluctant to share the true extent of their 
problems because of a decision by the grantee to "put the best face 
forward." Third, the evidence to justify a change might not have been 
conclusive at the time of the problem, and project staff were 
optimistic that they could still correct the problem later in the grant 
period. Finally, the large number of grants (30 to 35) that had to be 
monitored by each Federal program manager plus the additional workload 
of reviewing new applications and drafting revisions to program 
regulations meant there was little time to do more than cursory study 
of the grantees' reported activities. 

OVAE may want to consider two solutions to the problem of grantees 
not divulging serious implementation problems. First, OVAE program 
managers should discuss potential implementation problems with grantees 
at their kick-off meeting in Washington, D.C. The discussion should 
include both the nature of the problems encountered by other demon- 
strations (not just the Cooperative Demonstration Program) and the 
conditions under which it is important and acceptable to request 
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changes in the scope of the grant. Second, program managers should use 
the monitoring telephone calls to discuss both the accomplishments of 
the project and the problems (potential or actual) preventing expected 
accomplishments. If project directors are open about the problems they 
encounter, then the program manager will be better able to suggest 
solutions or put the project director in touch with other grantees 
having similar problems. This more open, communication however, will 
require program managers to become more active in the internal 
operations of the project. 
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B. Administering the Application and Grant Award Processes 



Instructions to Applicants 

The initial instructions to applicants are important for conveying 
information about program priorities. The instructions also can be 
useful in ensuring that applicants give sufficient forethought to their 
projects and avoid problems during implementation. The OVAE program 
staff has a certain amount of discretion in terms of the amount of 
information it provides in the application and the manner in which it 
is presented. 

Instructions to applicants can be more specific than those 
currently in effect. The study team examined similar regulations for 
demonstrations in both the U.S. Department of Labor (DOL) and the U.S. 
Department of Health and Human Services (DHHS). The instructions 
issued by the Office for Substance Abuse Prevention (OSAP) in DHHS 
offers an interesting model. Overall, the OSAP instructions are much 
more comprehensive and give a better understanding of how an applicant 
should respond to the grant announcement. As described earlier, the 
OSAP grant announcement provides a clear definition of what a logic 
model is a»^d a sample outline for conceptualizing and developing a 
demonstration grant using the components of a logic model. 

One of the OSAP grants announcement's major components is 
evaluation. The Anti-Drug Abuse Act of 1988 stipulates that priority 
should be given to demonstrations that employ appropriate strategies 
for evaluating the effectiveness of their proposed project. OSAP will 
only support projects with a "well-developed" evaluation plan. All 
evaluation plans are expected to include both process and outcome 
evaluations: 

Process evaluation is a quantitative and qual ititative 
description of the intervention, target population, and staff 
of a project from inception. Process evaluation should 
clearly and comprehensively document the relationship of the 
resources and program activities to the project objectives so 
as to permit mid-project adjustments as needed to optimize 
project implementation and ultimate replication. The 

er!c 



V-11 



evaluation plan should include a description as to how the 
components of a process evaluation have been or will be 
obtained/collected and maintained. The following are 
components of a process evaluation: 



1. Problem : the process by which the problem 
was defined as to (a) what is being demon- 
strated, (b) selection of risk factors to 
be addressed, (c) analysis of process by 
which population becomes at risk; 

2. Target Population : including demographic 
and other relevant characteristics, case 
finding and retention strategies; 

3. Goals and Objectives of the Pro.iect to be 
Evaluated : the process by which the goals 
and objectives of the project to be eval- 
uated were selected; 

4. Staffing Patterns : staff characteristics 
and qualifications including that of pro- 
ject director; supervision patterns; staff 
selection processes; staff activities and 
work schedule; 

5. Referral and Case Finding Patterns (if 
appl icable) : number, type, characteristics 
of r?^ferrals to and from project; partici- 
pating agencies in project including the 
development of inter-organizational link- 
ages with these agencies; 

6. Intervention : frequency, duration, type of 
contract; client flow, intervention mate- 
rials, manuals, staff training; staff and 
client perceptions of the interventions and 
objectives of the project; 

7. Cost Data : funding sources, cost per ser- 
vice, cost per client; 

8. Evaluation Procedures : monitoring instru- 
ments, need and risk assessment instru- 
ments; feedback mechanisms to director, 
staff, community representatives; and 

9. Generalizabilitv of Program Findings for 
Program Dissemination : Manuals and/or 
curricula that will be produced. 
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Outcome Evaluation: The evaluation plan should be 
detailed and clearly articulated. It should present an 
evaluation design appropriate to the project and of 
sufficient rigor to permit drawing valid conclusions 
concerning the effectiveness of the various intervention 
strategies. Outcomes variables should be derived from the 
logic model . 



At leust one grant demonstration program requires applicants to 
describe in detail the projected performance indicators for the 
project. The U.S. Department of Labor's "Disabled Program" requires 
advanced estimates of performance indicators similar to those used in 
the cost benefit portion of this evaluation: 



■ Placements . Indicate the number of trainees 
who will be trained and placed in unsubsi- 
dized employment upon completion of the ser- 
vices provided (which cannot be less than 
100); 

■ Average Cost Per Entered Employment . In 
relation to the total Federal costs proposed, 
indicate the expected average cost for each 
trainee who will be trained and placed into 
unsubsidized employment; 

■ Average Hourly Wage . Indicate the expected 
hourly wage that will be received by trainees 
upon completion of the program; and 

■ Projected Performance Indicators shall be 
provided on a quarterly basis and for each 
project site (Announcement SCA/DAA-92-001) . 



Timing of the Grant Award 

The timing of the notification of the grant award and the starting 
date for the award have important implications for project implemen- 
tation. Most academic institutions plan their staffing and activities 
according to the academic-year calendar, i.e., the academic year starts 
in the fall and ends in late spring or early summer. The institution 
may offer summer activities, but these are secondary to those conducted 
during the regular school year. Academic institutions plan in the 
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spring for the following year and try to have both activity schedule 
and staff assignments settled by May or June preceding the September 
start. 

Projects that start at the beginning of the academic year seem to 
be easier to implement than those starting mid-year. The FY1989 
grantees were notified of their awards in October 1989 and most 
negotiated award starting dates of January 1, 1990. (Two projects 
delayed their date until July 1990 because of existing Cooperative 
Demonstration grants that were still in progress.) The difficulty with 
a January start is that all available grantee staff are already 
assigned to other projects, are in the midst of those assignments, and 
are reluctant to shift to a new assignment. If the project goes 
outside the grantee to hire project staff, there are fewer qualified 
staff available. 

If new staff are hired, or even experienced staff reassigned, the 
planning for the project is taking place after other institutional 
events and programs have been established. Therefore, staff and other 
resources may not be available for use by the project as proposed. It 
might help the project to align the start of training activities with 
the start of the school year. 

The main exception to scheduling project start-up for the begin- 
ning of academic year is curriculum development projects that require 
significant amounts of instructor time. In this type of project, the 
start of the curriculum development phase should coincide with summer 
vacations when schools are out and teaching loads are light. In the 
summer, instructors have more time, are better able to concentrate, and 
will be able to integrate their new curriculum with their lesson plans 
for the next year. 

Although grantees have two to three months advanced notice of the 
grant start by virtue of entering into negotiations with the Depart- 
ment, grantees often do not start planning or hiring until the grant 
award officially starts. Several grantees commented that they knew of 
funding (Federal or State) that was canceled just before the award was 
made, and so the grantees no longer initiate work based on verbal 
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agreements. Grantees wait until the grant has officially started 

before they began to look for staff, rent space, assemble equipment, or 
negotiate contractual agreements with partners. For example, if a 

grant award is made on September 30th for a project start date of 

December 1st, the grantee will wait until December 1st to begin any 
activities. 

Length of the Grant 

The time period for the grant also has important implications for 
the activities that can be carried out and the ability to draw conclu- 
sions about outcomes from the grant. The time period for the Coopera- 
tive Demonstration grants was 18 months, with some projects requesting 
three- or six-month extensions. For grantees with no existing staff or 
training activities to assign to the project, between three and six 
months of the grant period had to be devoted to hiring staff, estab- 
lishing office and contracting procedures, and planning training. For 
grantees with existing staff or similar training activities already in 
place, startup costs were not as high but the training offered was 
often more extensive (part of at least a two year training program). 

Most grantees agreed that 18 months was not enough time to 
accomplish anything other than customized training. A more workable 
arrangement would dissaggregate the grant into two separate--but 
related— grants totaling 24 months. The Tirst grant would be a six- 
month planning grant during which the grantee could update the labor 
market information on which the training is justified. 

There can be as many as 18 months from the time the labor market 
information is collected and the award starts. During that time, local 
market conditions can change dramatically. Business can shut down, new 
employers move into the area, competing programs start, or the 
technology of the field can change. The planning grant also would 
allow the grantee time to hire the staff necessary to run the project. 
The advertising, interviewing, hiring, and orientation can take as long 
as four months. Once project staff have been hired, they will require 
an additional two months to establish routine office procedures. 
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Finally, the planning grant allows time for the grantee to negotiate 
specific roles and responsibilities with the private partner. If the 
proposed private partner is unable to fulfill commitments made during 
the application process, the planning grant then gives the grantee time 
to find an alternative partner(s). 

The second grant would fund actual training or other proposed 
acti 'ities for 18 months. Because grantees currently find it difficult 
to attract project staff if there is no certainty of long-term employ- 
ment, the operating grant would have to be virtually guaranteed unless 
their plans were not adequately developed. However, the certainty of 
24 months of funding must be balanced against the need for projects to 
demonstrate appropriate and efficient planning at the end of the 
planning grant. The OVAE program managers (and perhaps the original 
review panels) would be required to review and approve the operating 
plans before the second grant could be released. 



Section VI 
FINDINGS AND RECOMMENDATIONS 
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VL FINDINGS AND RECOMMENDATIONS 



This section summarizes the major findings from sections II, III, 
and IV, and answers the three evaluation questions framing the study. 



A. Did the Grant Applications Present a Clear 
and Coherent Design for a Project ? 



■ Of the 23 FY1988 applications reviewed, three 
were judged high in the likelihood of being 
successful (being both logical and 
plausible), ten were judged as moderately 
likely, and ten were judged less likely. The 
quality of the awarded applications improved 
in the second year. Of the 30 FY1989 
applications reviewed, nine were judged high 
in the likelihood of being successful (being 
both logical and plausible), 16 were judged 
as moderately likely, and five were judged 
less likely. 

■ Key terms in the applicat'on package and 
program regulations were not operationally 
defined, resulting in wide disparities across 
projects. 

■ Applications frequently did not state clearly 
what was to be demonstrated. 



Based on these findings and other information collected during the 
evaluation, the study team recommends that: 



1. Program regulations should be more 
precisely worded to convey a clear meaning 
of key terms such as "demonstration" and 
"hi jh-technology"; 

2. Applicants should be instructed to clearly 
show the logic and plausibility of their 
project designs; 

3. If logic and plausibility are important to 
the Department, they should be assigned 
points in the evaluation criteria (much 
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like the points assigned for organization 
capacity); and 

The panel reviewing the applications should 
include experts familiar with the high 
technology field or activities being 
proposed. Such experts could identify 
activities which are not feasible within 
the time frame or resources proposed, A 
bank of experts can be identified and 
tentatively invited prior to the receipt of 
the applications, and those with direct 
expertise can be appointed to the panels 
once the technical areas have been 
identified. 



VI~3 

B. Were the Project Designs Successfully Implemented ? 



■ FY1988 grantees were generally surprised that 
they were funded and were not prepared for 
the immediate start of the project. FY1989 
grantees were less surprised because of 
better monitoring of the award process (and 
several were FY1988 grantees). 

■ Grantees who were already operating similar 
projects were able to implement their pro- 
jects faster and more successfully than pro- 
jects starting from scratch. The availabi- 
lity of existing partnerships, staff, and 
institutional support was a key factor in the 
project's success. 

■ Local labor market conditions significantly 
worsened after the time the applications were 
submitted and made it difficult to implement 
the proposed internship, job placement, and 
recruiting components. It appears that 
applicants may not have fully disclosed or 
understood local economic conditions in their 
appl ication. 

■ Grantees who had no prior working relation- 
ship with the proposed partner{s) had diffi- 
culty getting cooperation or support from 
that partner(s). The problems in obtaining 
cooperation from the partner organization 
were exacerbated by local labor market 
conditions. 

■ Projects with larger numbers of private part- 
ners used those partners in less intensive 
ways, while projects with fewer partners used 
them more intensively. Projects with fewer, 
more involved partners tended to be more 
successful . 

■ Grantees targeting hard-to-serve populations 
tended to shift their focus to less chal- 
lenging students once they realized the dif- 
ficulty of what had been proposed. A few 
projects added enhanced basic skills training 
for the hard-to-serve students they did 
recruit. Several projects were unrealistic 
in their expectations of clients or their 
assumptions about the needs of employers. 
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■ Projects providing customized job training 
were more successful in training the proposed 
numbers of students than were other types of 
projects. In some projects, however, the 
customized training subsidized training the 
partner would have done even without the 
project, 

■ The training provided by most projects was of 
short duration. Except for a few multi-year 
training efforts (of which the grant funded 
only a portion, most students received fewer 
than 100 hours of training. The short 
duration of training reflects the fact that 
most programs involved some type of custom- 
ized training, which in t rn reflects the 18- 
month time frame of the grant. 

■ Staffing shortages in OVAE resulted in 
grantees receiving less technical assistance 
and guidance from project officers than 
initially planned. 

■ Most of the third party evaluations of pro- 
jects produced little feedback to grantees 
and limited outcome information at the end of 
the project. 

■ Very few of the projects involved students in 
secondary schools. It was not possible to 
determine whether the low representation was 
due to lack of interest or awareness by 
secondary schools or whether it was a result 
of the grant review process. The low 
involvement of secondary schools resulted in 
few lessons about public private cooperation 
at this level . 



Based on these findings and other information collected during the 
evaluation, the study team recommends that: 



1. Grant awards should be made in the spring 
for project starts in the summer because of 
the greater availability of grantee staff, 
reduced training demands by other programs, 
and more opportunities for planning. 
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2. The applicant's justification of labor 
needs should address the geographic area 
immediately surrounding (i.e., within 
reasonable commuting distance for the 
students' being trained) and use current 
labor market data. 

3. Applicants should state in detail the role 
and responsibilities of the private part- 
ner(s) and include a statement from the 
partner agreeing to accept this role. 
Applicants should be encouraged to pursue 
only a few partners but have those partners 
much more heavily involved. 

4. Grantees should be encouraged to request 
changes to the scope of work in response to 
changes in the local labor market and other 
conditions. Some grantees made extensive 
ad hoc changes in the project design while 
others felt they were not allowed to 
deviate from the application. 

5. The Department should consider awarding a 
six month planning grant combined with an 
18- to 24-month implementation grant. 

6. Grantees should be held more accountable 
for meeting the objectives stated in their 
application or revised scope. Objectives 
should be stated in operational terms that 
can be measured by the third party evalu- 
ator. 

7. OVAE staff should hold additional meetings 
of all project directors as a group. These 
meetings should address management, evalu- 
ation, and reporting issues. OVAE may want 
to consider a program for on-going techni- 
cal assistance to grantees, much the way 
other Federal agencies do. 

8. The Department should formulate procedures 
for ensuring greater participation by 
secondary schools. 
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C. Are Project Costs Reasonable in Relation to Pro.ject Outcomes 



■ Planning and administrative costs accounted 
for between two percent and 35 percent of 
total project costs, with most projects 
averaging 35 percent. 

■ The cost per student trained and per hour of 
training was similar across projects. 

■ Project costs were greatly affected by the 
monetary value assigned to the equipment and 
other contributions from private partners, 
with no assurance of standard values across 
projects. 



Based on these findings and other information collected during the 
evaluation, the study team recommends that: 



1. Applicants should better substantiate the 
value assigned to in-kind contributions. 

2. The contributions from the private sector 
should be clearly distinguished from 
contribution^ from the grantee institution. 

3. Grantees should be held to the same non- 
supplanting requirements of other Perkins 
Act programs. 

4. Applicants should be told at the start of 
the grant what cost and performance data 
they will be expected to provide at the end 
of the grant. 
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PUBLIC LAW 98-524 ESTABLISHING THE 
COOPERATIVE DEMONSTRATION PROGRAM 



A-1 

"Pabt S^DKMONvnunoN Pkogeams 

''Subpart 1— Cooperative Demoiuitration Programs 

**PWOQMAU AUTHOaiZEO 

Gnnu. "Sac. 411. (a) Frutn the amounts available for this p^rt under 

c^trmcu with section 451 for each fiscal year, the Secretary is authorised to carry 
20 U9C24n directlT or through grants to or contracts with State and local 

Po€t. p. 2479. educational agencies, pcetaeoondary educational institutions, insti- 
tutions of higher education, and other public and private agencies, 
organizations, and institutions, programs and projects which sup- 
port- 
ed) model programs providing improved access to quality 
vocational education pr 'jgrams for those individuals described 
Anu, p 2450. in sectioQ 201(b) of this Act and for men and women seeking 

nontraditkmsl occnpatiooa; 

"(2) examples of successful cooperation between the private 
sector and public agencies in vocational education,- involving 
empioyefa or consortia of employeca or labor organizations and 
building trade councils, and StaU boards or eligible recipienU 
designed to deroenatrate ways in which vocational education 
and the private sector of the ecoooray can work together effec- 
timy to assist vocatiimai educatioa students to attain the 
advanced level of skills needed to make the transition from 
school to productive employment, including— 

"(A) work e x perience and apprenticeship programs; 
(B) transitional worksite job training for vocational edu- 
cation itttdenta which is related to their occupational goals 
and closely linked to c las sroo m and laboratory instruction 
provided by an eligible recipient; 

"to plaoMnent services in occupations which the students 
are preparing to enten and 

"(D) where practical, projecU (such as the rehabilitation 
of pubbc schools or housing in inner cities or economically 
„^ depressed rural areas) that will benefit the publk; 

(3) programs to overco m e national skill shortages, as desig* 
nated by the Secretary in cooperation with the Secreury of 
Labor. Seoetary of Defense, and Secretary of Commertre; and 

(4) such other activitica which the Secretary may designate 
which are related to the purposia of this Act 

insUtutiOQai and on^e^ trauung, supportive services jiuthorized 
by this Act, and such other necessary assistance as the Secretary 
deterrauiea to be necessary for the successful completion of the 
project. 

"(2) Not less than 25 percent of the cost of the demonstration 
progxams authorized by this subpart shaU be provided by the reripi- 
ent of the grant or contract, and such share may be in the form of 
cash or in-kind contributiona* including facilities, overhead, person- 
ntl and eqmpme&t fairiy valued 
"(c) All prpgrama assisted under this section shsU be- 

(1) of direct service to individuals enrolled in such programs: 
and 

«pable of wide replication by service pnmders. 
(d) The Secretary shall disseminate the resuiu of the programs Public 
and projecca assisted under this section in a manner designed to avuiabUitr. 
improve the tnining of teachers, other instructional personnel 
counsellors, and administrators who are needed to carrv out the 
purposes of thia Act ^ 

"^^U^Sf 7^ •ft*^ ^ of enactment of the 

Carl p. Periana Vocational Education Act, the Secretary of Labor 
and the Secretary of Education shall develop and implenwnt a olan 
for greater coordination between vocatioaal educatioa pronaim and 
approticeship trainu^ ppqgrama, Linkagea b e t wee n sudiprograms 
b» J M tahhshari reiatinf to apprentioeediool progrmnTand 
preappientioeship programa, and program evaluation and nerfbrm- 



and progxuia of related uutniction). Tho SeoMtriMshairMUibljiih 
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34 CFR CH. IV SECTION 412 REGULATIONS GOVERNING 
THE COOPERATIVE DEMONSTRATION PROGRAM 
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PAIT 412— COOPERATIVE 
DEMONSTRATION PROGRAM 

412.1 Wli*t is ihr Co<i|>orntivr Drmonslrn. 
lion Program? 

412.2 Who is elltlblr to apply for an aaard 
under tliis proitram? 

412.3 What refulattons apply lo this pro- 
Cram? 

412.4 What definitions apply to this pro- 
irmm? 

Ui^pmi %^Whmf KMt •! ActlvHi«t 0*#« fli« 
S«<rM^ Attist Mndf TWs frm^rmmJ 

412.10 What types of projects may be 
funded? 

412.11 How does the Secretary establish 
priorities for this proxram? 

Subject 0_H«w 090% lh« S^crvtcry Mtth« • 
Cf«f«f? 

412.30 How cices ihc Sfcrrtarv evaluatt* an 
aopilcatioii? 

412.31 What selection criteria does ttir St»c 
retary iise? 

412.40 What cost sharing requirement is 
Impoicd under this procram? 
Autmohity: Sec. 411 of the Carl D. Per- 
kins Vocmiional Education Act. 20 U.S.C. 

2411. as t-nactrd bv Vuh I. 9H JS24. iinle« 
otherwisr notexi. 

Sowacc 50 FR 33260. Aug. 16. 1985. unleiw 
otherwise noted. 

Subpart A~G«n«ral 

<J 412.1 What is the ( ooperative Demon, 
siration Program? 

(a) The Cooperative Detnonstration 
Program provide*; fitianrial xssmancr 
for— 

(1) Model projects providing im- 
proved access to quality vocational 
cdttcation programs for certain indi- ' 
vtduals: 

(2) Projects that aro examples of 
successful cooperation between the 
private sector and public agencies In 
vocational education: 

(3) Projects to overcome national 
skill shorUges; and 

<4) Other activities which the Secre- 
tary may designate that are related to 
the purpose of the Act 

(b) Projecu eligible for assistance 
are desctlbed in § 412.10 

<A«thority:See.4IHai:20b.SC 24il<jin 20 / 



S 412.2 Who in eligible to upply f„r an 
■ward under thin program? 

The following are eligible to apply 
for an award under thi.^ program: 

(a) State edticational agencies 
(o£iAs ). 

(b) Local edticational agencies 
(LCAs). 

(c) Postsecondary educational insti- 
tutions. 

(d) Institutions of higher education 

(e) other public and private agen- 
cies. organizations, and institutions. 

< Atiihonty: Sec. 41 ha*. 20 U.S.C. 241lia)i 

9 II2.3 What regulationji iippiv to tht<i pro- 
gram? 

The following regulations apply to 
the Cooperative Demonstration Pro- 
gram: 

40<r* regulations in 34 CFR Part 

<b) The regulations in this part. 
<Authoray: Sec. 411: 20 US.C. 2411) 

^ II2.4 What dennttionft mppiy to thtK pn>. 
gram? 

The definitions in 34 CFR 400 4 
apply to the Cooperative Demonstra- 
tion Program. 

< Authority: Sec. 4 11; 20 U.S.C. 2411) 

SubfMrt i~Wh«t Kifidt mi ActivHlM 
Oo«i th« S«cr«tory Assist Und«r 
This ProfrMin? 

^412.10 What iyptn of projeetn may be 
funded? 

(a) The SecreUry may support dl- 
rectly or through grants, cooperative 
agreements, or contracu the following 
types of proiects: 

(I) Model projecu providing im- 
proved access to quality vocational 
education programs for— 

(i) Handicapped individuals; 

(II) Disadvantaged individuals: 

(III) AdulU who are In need of train- 
ing and retraining: 

(iv> Individuals who are single par- 
enU or homemakers: 

(V) Indlvldua>a who participate In 
programa designed to eliminate sex 
blag and stereotyping In vocational 
education; 
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(vi) Criminal offenders who are serv- 
ing in a correctionai Institution: and 

(vii) Men and women seeking non- 
traditional occupations. 

(2Hi) Projecta that are examples of 
successful cooperation between the 
private sector (including employers, 
consortia of employers, labor organiza* 
tiona, and building trade councils) and 
public agencies in vocational educa- 
tion, including State boards and eligi* 
ble recipients. 

(11) The projects described in para* 
graph (a)(2)(i) of this section must be 
designed to demonstrate ways in 
which vocational education and the 
private sector of the economy can 
woric together effectively to assist vo- 
cational education students to attain 
the advanced level of skills needed to 
make the transition from school to 
productive employment, including— 

(A) Work experience and apprentice* 
ship projects; 

(B) Transitional worksite Job train* 
ing for vocational education students 
which is related to their occupational 
goals and closely linked to classroom 

and laboratory instruct lun proviin h m 
an eligible recipient: 

(C) Placement services in occupa- 
tions which the students are preparing 
to enter and 

(D) Where practical, projects that 
will benefit the public, such as the re- 
habilitation of public schools or hous- 
ing in innsr cities or economically de- 
pressed rural areas. 

(ill) The projects described in para- 
graphs (a)(2) (1) and (ii) of this section 
may include institutional and on-the- 
job training, support services author- 
ized by the Act, and such other neces- 
sary assistance as the Secreury deter- 
mines to be necessary for the success- 
ful completion of the project. 

(3) Projects to overcome national 
skill shortages, as designated by the 
Secretary in cooperation with the Sec- 
reury of Labor, Secreury of Defense, 
and SecreUry of Commerce. 

(4) Such other activities which the 
SecreUry may designate which are re- 
lated to the purposes of the Act. 

(b) All projects assisted under the 
Cooperative Demonstration Program 
must be— 

(1) Of direct service to the individ- 
uals enrolled: and 

(2) Capable of wide replication by 
service providers, 

cAuthoritr Ste, 411 (a)r<b). (o: 20 U.S.C. 
2411 (a). (t>). (oi 



9 412.11 How 4otM the Secretary entafalixh 
priohtiea for this program? 

(a) The Secretary may announce 
through one or more notices published 
In the FcocKAt Rccisrot the priorities 
for this program (including any na- 
tional skill shortages to be addressed) 
If any, from the types of projects de- 
scribed in § 412.10. 

(b) The Secreury may esublish a 
separate competition for one or more 
of the priorities selected. If a separate 
competition is esublished for one or 
more priorities, the Secretary may re- 
serve all applications that relate to 
those priorities for review as part of 
the separate competition. 

(Authority: Sec. 411: 20 U^.C. 3411) 

Subpart C^(Keserve4i] 

Siibport D — Hew Does the Secretory 
Moke o Gront? 

0 412.30 How doM th« SeereUry evaluate 
an application? 

(a) The SecreUry evaluates an appM- 
cation for a grant or cooperative 
agreement on the basis of the criteria 
in f 412.31. 

(b) The SecreUry may award up to 
100 points. Including a reserved IS 
points to be distributed in accordance 
with paragraph (d) of this section, 
based on the criteria in § 412.31. 

(c) Subject to paragraph (d) of this 
section, the maximum possible points 
for each criterion is indicated in pa- 
rentheses after the heading for each 
criterion, 

(d) For each competition, as an- 
nounced in a notice published in the 
Feocral Registck, the Secreury may 
assign the reserved IS points among 
the criteria in §412.31. 

< Authority. Sec. 4 11 : 20 U.S.C. 24 U ) 

t Approved by Ihe OiUcr of M:inaR«fncni 

and BudKct iindrr control iitimb^r 1830- 

0013) 

^112.31 What selection criteria dtftn the 
Serretary uie? 

The Secretary uses the followini? se- 
lection criteria in evaluating each ap- 
plication: 

(a) Need. (IS points) 

(1) The Secretary reviews each ap- 
plication for information that shows 
the need for and the soundness of the 
rationale for the project. 
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(2) The Secretary looks for Jnforma- 
lion that shows— 

<l> A clear description of the need 
for the proposed project: 

Ml) Specific evidence of the net»d for 
the project: 

Mli) A description of any ongoing 
and planned activities in the communi- 
ty relative to the need, including if 
appropriate, the relationship of any 
local, regional or State economic do- 
vo!opm<»nt plan: 

Mv) Kvidonce thai demonstrates the 
vocational irainitig to be provided is 
designed to meet current and project- 
ni orriipaf tonal nf»f*<is: 

(V) A clear sutement of what the 
project seeks to demonstrate; and 

(VI) Evidence that the project Is 
likely to serve as a model In the 
future. 

(b) Plan 0/ opemtioru (20 points) 

(1) The Secretary reviews each ap- 
plication for Information that shows 
the quality of the plan of operation 
for the project. 

(2) The SecreUry looks for informa- 
tion that shows— 

(I) High quality In the design of the 
project: 

<il) An effective plan of management 
that ensures proper and efficient ad- 
ministration of the project: 

(III) A clear description of how the 
objectives of the project relate to the 
purpose of the program: 

(Iv) The way the applicant plans to 
use lU resources and personnel to 
achieve each objective: and 

(V) A clear description of how the 
applicant will provide equal access and 
treatment for eligible project partici- 
pants who are members of groups that 
have been traditionally underrepre- 
sented. such as— 

(A) Members of racial or ethnic mi- 
nority groups: 

(B) Women: 

(C) Handicapped persons: and ' 

(D) The elderly. 

(c) Quality of key personneL (10 
points) 

(1) The Secretary reviews each ap- 
plication for information that shows 
the qualifications of the key personnel 
the applicant plans to use on the 
project. 



UnVfl^^ Secretary looks for informa- 
tion that shows— 

(I) The qualifications of the project 
director (if one Is to be used)- ^ ^ 

(ii) The qualifications of eich of the 
other key personnel to be used in the 
project; 

f ^rfin T*'' """^ "'^'^ P«"on re- 
,fi\ J P*'^«''aphs (c)(2) (i) and 

prii«tl and"^"°" '° ^»'« 

(iv) The extent to which the appli- 

°^ nondiscriminatory 
♦•mployment practices, encourages ap- 
plications for employment from per- 
h°.ul members of groups that 

s.:n,nH''""K ''»d'l"'"«'^"y underreprc 
sentcd. such a.s— 

(A) Members of racial or ethnic mi- 
nority groups: 

(B) Women: 

(C) Handicapped persons: and 
<D)The elderly. 

(3) To determine personnel qualifi- 
cations, the Secretary considers expe- 
rience and training, in fields related to 
the objectives of the project, as well as 
other Information that the applicant 
provides. 

(d) Budget and cost tffectiveness. (10 
points) 

(1) The Secretary reviews each ap- 
plication for Information that shows 
the project has an adequate budget 
and is cost effective. 

(2) The SecreUry looks for informa- 
tion that shows— 

(i) The budget for the project is ade- 
quate to support the project activities: 
and 

(II) Costs are reasonable In relation 
to the objectives of the project. 

(e) Evaluation plan, (5 points) 

(I) The Secretary reviews each ap- 
plication for information that shows 
the quality of the evaluation plan for 
the project. 

CROU-RKmeiirt. S«» 34 CPR 75 590 
< Evaluation &y the trantcf»> 
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(2) The Secretary loofci for informa. 
tion that shows methods of evaluation 
that are appropriate for the project 
and. to the extent possible, are objec- 
tlve and produce data that are quantl- 
flable. 

(f) Adequacy of resources, (5 points) 
(1) The Secretary reviews each ap- 
plication for inforination that shows 
that the applicant plans to devote ade- 
quate resources to the project. 

<2) The Secretary looks for informa- 
tion that shows— 

(1) The facilities that the applicant 
plans to use are adequate: and 

(ii) The equipment and supplies that 
the applicant plans to use are ade- 
quate. 

(g) Prixmie sector irivolvement (5 
points) 

(1) The Secretary reviews each ap- 
plication for information that shows 
the involvement of the private sector 

(2) The Secretary looks for informa- 
tion that shows— 

(I) Private sector involvement In the 
Planninc of the project: and 

(ii) Private sector involvement in the 
operation of the project. 

(h) Employment opportunities. (5 
points) 

The Secretary looks for information 
and documentation of the extent to 
which trainees will be employed In 
jobs related to their training upon 
completion of their training. 

(I) Dissemination, (10 points) 

(I) The Secretary reviews each ap- 
plication for information that shows 
that the applicant has an effective and 
efficient plan for disseminating infor- 
mation about t he demonstration 
project, including the results of the 
project and any specialized materials 
developed by the project. 

(25 The Secretary looks for informa- 
tion that shows*- 

(i) High quality in the design of the 
dissemination plan and procedures for 
evaluating the effectiveness of tlie dis- 
semmation plan: 

(ID A description of the types of ma- 
terials the applicant plans to make 
available and the methods for making 
the materials available: 

Uil) Provisions for demonstrating 
the methods and techniques used by 
the project: 

(iv> Provisions for assisting others to 
adopt and successfully implement the 
project or methods and techniques 
used by the project: and 

(V) Provisions for publicizing the 
findings of the project at the local 
State, or national level. 

(Authority: Sec. 411: 20 U.S.C. 2411) 
(Approved by the Office of Manacrinrni 
tnd Budget under control luimber 1830- 
00I3> 



Subporf E^Whaf Condltlont Must B« 
M«f by o K«cipi«nt? 

n 412.10 What eofii ^hnrinit requirement h 
imp<Hied under (hix proftmm? 

t^^l A recipient shaM provide not less 
than 25 percent of the cost of the 

under thk^ program. 

-j^'^™ ^^^^ Subpart G of 

34 CPR Part 74, the non-Federal share 
may be in the form of cash or in-kind 
contributions, including the fair 
market value of facilities, overhead 
personnel, and equipment. 
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FY1988 GRANTEES REVIEWED IN THE EVALUATION 



Grantee 



Project 



Central Community College- 
Platte Campus 



Competency-Based Modular Assess- 
ment and Training for Maintenance 
Technicians in Manufacturing 



Division of Vocational Edu- 
cation Services 
State Department of Education 
Montgomery, Alabama 



Student Apprenticeship Link- 
age in Vocational Education 



El Paso Community College 
El Paso, Texas 



CAREER Program: Career Assessment, 
Remediation, Education, Employment, 
and Re-entry 



Francis Tuttle Vocational* 

Technical Center 
Oklahoma City, Oklahoma 



High Technology Partnership 
Project 



Greenville Technical College 
Greenville, South Carolina 



Project TEAM: Technical 
Education Advancement Modules 



Hampden County Employment 
and Training Consortium 
Springfield, Massachusetts 



Project CREATE: Cooperative 
Resources to Enhance Access to 
Jobs Through Technical Education 



Indian Hills Community College 
Ottumwa, Iowa 



Indian Hills Cooperative Demon- 
stration Program 



Indiana University of 

Pennsyl vania*# 
Reschini House Indiana, 
Pennsylvania 



Northwestern Pennsylvania 
Cooperative Demonstration for 
Technical Updating 



*responded to telephone survey 
•received site visit 



2U 



B-2 



Northampton Community College*^ 
Bethlehem, Pennsylvania 



Turn-key Surface Mount 
Training Program 



Parkland College 
Champaign, 111 inois 



Advanced Certification Program 
for Computer Graphic Specialists 



Portland Community College 
Portland, Oregon 



Women in Education for Appren- 
ticeship and Non-Traditional 
Employment 



Postsecondary Vocational- 
Technical Education 
Concord, New Hampshire 



New Hampshire Automotive 
Education Collaborative 



Richland School District*^ 
Kennewick, Washington 



Materials Technology: The Common 

Core Skills That Are Shaping the Future 



Skyline College*^ 
San Bruno, California 



Toyota/Skyline Partnership 
for Automotive Technician 
Training 



Southern Growth Policies 
Board 

Research Triangle Park, 
North Carol ina 



Consortium for Manufacturing 
Competitiveness 



Toledo Public Schools 
Toledo, Ohio 



Industrial Automation Mechanic 
Model Curriculum 



University of North Dakota- 
Lake Region 
Devils Lake, North Dakota 



Flight Simulator Maintenance 
Technician 



University of Wisconsin- 
Stout*# 

Menomonie, Wisconsin 



Implementing a High-Tech Train- 
ing Model for Rural Based Busi- 
ness and Industry, Technical 
Colleges, and Local and State 
Education Agencies 



*responded to telephone survey 
• received site visit 
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Valencia Community College*^ 
Orlando, Florida 



Valencia Community Conege*^ 
Orlando, Florida 



B-3 



A Model, Repli cable Advanced 
Manufacturing Demonstration 
Project 

Film Production Technology 
Training Program 



Ventura Community College 

District*^ 
Moorpark College 
Moorpark, California 

Waubonsee Community College 
Sugar Grove, Illinois 



Non-college Bound Student 
Demonstration Project - Electronics 
/Laser/El ectro-optics 



A Comprehensive Development Plan 
in Office Skills 



Yakima Valley Community 

College 
Yakima, Washington 



Extending Health Training and 
Services to Rurally Isolated 
Populations in a Depressed Area 



*responded to telephone survey 
•received site visit 
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FY1989 GRANTEES REVIEWED IN THE EVALUATION 



Grantee 



Project 



Bronx Community College*^ 
Bronx, New York 



A Model Program Demonstrating 
A Public/Private Sector 

Cooperative Training Program 
in Radiologic Technology 



Center for Occupational 

Research and 

Development (CORD)*# 
Waco, Texas 



Semiconductor Manufacturing 
Technology Program 



Clackamas Community College*^ 
Oregon City, Oregon 



Precision Manufacturing 
Technology Project 



Fox Valley Technical College 
Appleton, Wisconsin 



A "Systems" Approach to Providing 
Cost-Effective Training and Tehnical 
Assistance in CIM to Small 
Manufacturers 



Hampden County Employment 

and Training Consortium*^ 
Springfield, Massachusetts 



Project: High-Tech '90 



Home Builders Institute*^ 
Washington, D.C, 



Training and Certification 
Program for Smart House 
Instal 1 er/Techni ci ans 



Howard Community College*^ 
Columbia, Maryland 



Enhanced Re-entry Nurses 's 
Education Using Interactive 
Videodisc Assessment/ Instruction 



Indian Hills College 
Ottumwa, Iowa 



A Model to Provide Degree-bearing 
Training to Non-traditional Students 



*respond to mail survey 
•received site visit 
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Illinois Eastern Community 

College*^ 
Olney, Illinois 



Training in Semiconductor 
Technology 



LTV Steel Company 
Whiting, Indiana 



Electronic and Instrument Repair 
Technician Apprenticeship 



Luzerne County Community 
College** 



Advanced Technology Center 
Computerized Numeric Control 



Nanticok^, Pennsylvania 



Cooperative Demonstration 
Training Program 



Nebraska Department of 

Labor** 
Lincoln, Nebraska 



Rural Allied Medical Business 
Occupations Program 



North Clackamas School *^ 
District #12 
Milwaukee, Oregon 



Student Cooperative Training 
Units Program 



Partners for American 

Vocational Education (PAVE)*^ 
Alexandria, Virginia 



Business-Education Venture for 
Health Care Occupations Training 



Southwestern Community*^ 

Col lege 
Chula Vista, California 



Comprehensive Aerospace 
Manufacturing Technology 
Program 



State Center Community*^ 

College District 
Fresno, California 



Advanced Technology Center 



Waubonsee Community College*^ 
Sugar Grove, Illinois 



Survival Skills for Office 
Technicians 



*respond to mail survey 
•received site visit 
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West Virginia Department 

of Enucation*# 
Charle:ton, West Virginia 



Cooperative Demonstration of 
High Technology Programs in 
Computer Language and Health 
Occupations 



West Virginia Northern 
Community College** 
Wheeling, West Virginia 



High Technology Training: The 
Other Half of the Equation 



*respond to mail survey 
•received site visit 



21 
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RAW SCORES OF PROJECT PLAUSIBILITY FOR 
FY1989 PROJECTS 
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Appendix D 

BRIEF DESCRIPTIONS OF FY1988 GRANTEES 
SURVEYED OR VISITED 



D-1 

Richland School District, Kennewick. Washington 

The original plan, largely implemented, was to introduce a one-year 
course in materials science and technology (MST) to seven high schools 
and one community college. The MST course curriculum had already been 
developed by a teacher at Richland High School with support and 
technical assistance from Battelle Northwest Laboratories (the private 
partner) • The course was a ISO^hour, hands-on, science and vocational 
course about glass, ceramics, metals, composites, and wood. Two 
teachers from each of the seven high schools were trained in a three 
and a half week summer workshop. They adapted MST outline and workbook 
to their own school and local needs, and trained 237 students during 
the 1989-90 school year. One site dropped out at the mid-year point 
because of school construction and staff turnover. 

Northampton Community College. Bethlehem. Pennsylvania 

The National Training Center for Microelectronics at NCC proposed 
to providing local manufacturing companies (the private partners) with 
customized job training in surface mount technology (SMT). During the 
grant period, the Center expanded its existing training program in SMT, 
enhancing four existing courses, creating seven new courses, acquiring 
new equipment, and producing a national teleconference. The project 
trained 233 employees of seven microelectronics firms during the grant 
perioJ at reduced or no cost to the companies. The project also 
reached an estimated 2000 employees at 18 locations nationwide through 
two four-hour teleconferences, one of which was interactive. The 
subject of the conferences was "Packaging in the 1990s," and videotapes 
of the teleconference were sold to private companies and donated to 
other universities. The project was generally implemented as proposed, 
but trained only about half as many students as planned because 
companies enrolled fewer students than predicted. 

Valencia Community College, Film. Orlando. Florida 

The VCC staff established as proposed film production technology 
program to train students for jobs in the growing local film industry. 
With technical support from Universal Studios (the private partner), 
VCC developed the program's curriculum and implemented a 15-week course 



offered three times during the grant. A total of 135 students were 
trained in stagecraft, sound, set construction, camera/editing, and 
post production. Together, the three sessions produced a full-length 
feature film entitled "Sealed With A Kiss." The project was 
implemented as outlined in its proposal. 

Valencia Community College, Manufacturing, Orlando. Florida 

VCC proposed providing a local manufacturing company with 
customized job training in automated manufacturing technology. VCC was 
already working with Stromberg-Carl son. Inc. (the private partner) 
under a State of Florida grant to help Stromberg-Carl son introduce high 
technology-based manufacturing processes. During this project, VCC 
trained 565 Stromberg-Carl son employees with the curriculum designed 
under the earlier grant. A total of 26 classes were offered in 17 
different courses; the courses averaged 65 student contact hours. The 
project was implemented as proposed, although 30 percent of the 
students dropped out prior to graduation due to other demands on their 
time. 

University of Wisconsin - Stout, Menomonie. Wisconsin 

The project proposed implementing a "model" high technology 
training program in three technical colleges and 12 high schools in UW- 
Stout's service area. Project staff conducted an initial four-week 
summer workshop and trained teachers from the participating secondary 
schools and postsecondary technical institutes how to conduct local 
needs analyses and to develop module curriculum. Teachers at each 
school interviewed local industry officials to determine high-tech 
training and skill needs and to develop course modules. Teachers spent 
the 1989-90 school year preparing their module(s) and testing them in 
classes. The project planned to complete 41 modules—ranging from a 
few hours to a full semester of instruction. The project was 
implemented as proposed except that teachers at the participating 
technical colleges did not have time to begin training employees of the 
private sector partners. 



Skyline CoTmnunitv College, San Bruno> California 

The vocational division at the college proposed a joint training 
program for service technicians with Toyota Motors Sales (the private 
partner). The program, called T-TEN, included 16 weeks of formal 
instruction per year for three years and part-time work at a reduced 
wage in Toyota dealerships. Toyota provides financial incentives to 
the college and to students for implementing and graduating from the 
program. T^f project was underway, with state finance, prior to the 
Cooper tivt; Demonstration grant. Potential students were recruited 
through newspaper ads and selected by dealership personnel during an 
annual meeting. Tre college began the process of obtaining national 
certification for i.s automotive program (NATEF). At the end of two 
years, a total of 1/ students were enrolled in the program and two 
Skyline i'^itructors had received extensive Toyota training and 
developed the training curriculum using Toyota materials. The project 
was laroeiy implemented as planned, except 1) there were fewer 
participants because there were fewer job opportunities with Toyota 
de^lerships, and 2) most students ended up working full-time because 
they needed the money and dealers needed the staff. 

Ventura Community College District - Mooroark College. Mooroark. 
Cal ifornia 

The original plan was to establish a coordinated high school- 
college program in electronics and laser/electro-optics for at risk 
students in 8 high schools. In addition, the local business/labor 
council (BLC~the private partner) would arrange for field trips to 
potential employers, transportation among sites, counseling, and other 
activities. The project curriculum was to be developed at the college. 
The initial plan was modified considerably over the grant period. A 
summer remedial basic skills program was not held. High school 
instructors received informal training from the college as needed to 
implement the program. Planned inter-school transportation was 
simplified. The BLC role diminished substantially. As jobs in laser- 
optics decreased, college attendance became a more likely student 
outcome. In the end, however, the schools adopted the college- 
developed program, 52 students from 8 high schools completed the 



coordinated instructional program, more are now enrolled in the 
college* Students received instruction at 4 of the schools for 4 days 
a week and at Moorpark one day a week. 

Indiana University of Pennsylvania. Indiana. Pennsylvania 

The initial goals of this project were ambitious----to establish 
collaborative technical training between Indiana U, county vocational 
schools, other postsecondary institutions, private trade schools, 
regional economic development agencies and the private sector. 
Problems in coordinating with the regional economic development agency, 
scaled back goals* In the end, the project delivered a variety of 
short-term training programs to the employees of smaller manufacturing 
and other firms in the area (the private partners) at no cost to the 
companies or employees* Courses varied from basic math to the use of 
sophisticated computer controlled machinery* Most courses were offered 
by the county vocational schools arcl approximately 648 students 
received training or attended product demonstrations. Thirty-three 
classes were organized* 

Francis Tuttle Vocational-Technical Center, Oklahoma City, Oklahoma 
Based on telephone survey only: The goal of this project was to 
increase enrollment in the High Tech Center within the institution by 
providing academic remediation to adults who would not otherwise 
qualify, and to extend formal instruction with internships. This 
project was largely implemented as planned. A recruitment campaign was 
undertaken and a self-paced learning lab installed. To attract 
students, the project provided tuition reimbursement for 220 students 
without regard to financial need. Eighty percent of the Center's 
students used the lab. Internships with stipends paid from the grant 
were provided to 20 students (sites were the private partners). The 
project recorded a substantial increase in enrollments and a dropout 
rate of 30 percent. 
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Bronx Community College 
Bronx, New York 



Bronx Community College (BCC), one of the City University of New 
York (CUNY) Colleges, is located in the hub of urban activity. The 
Radiologic Technology (RT) Program, funded by the Cooperative 
Demonstration Program grant, was administered through the Physics 
Department at BCC. 

The goals of the program were to provide students with employment 
opportunities and to provide hospitals and other medical facilities 
with trained radiographers in order to increase their employee pool. 
More specifically, as outlined in the proposal, the program was to: 

■ Help alleviate the current shortage of 
radiologic technologists at Columbia- 
Presbyterian Hospital; 

■ Help improve the recruitment of Blacks and 
Hispanics into radiologic technology; and 

■ Aggr ssively recruit new workers from 
nontraditional labor pools including the 
unemployed, recent immigrants, and older 
workers. 

The work of the partnership would also involve conducting 
placement tests, arranging for student stipends, offering student 
advisement, arranging for the program to be accredited, preparing the 
trainees for licensure, assisting informally in placing students in 
jobs, and preparing a manual documenting the BBC training model for 
dissemination. 

BCC teamed up with Columbia-Presbyterian Hospital, as planned. 
After the grant was obtained a second partnership, with Hontefiore 
Hospital, was formed. The hospitals and BCC are all located in New 
York City. The partnership began with the Cooperative Demonstration 
Grant. A hospital workers, union. Hospital League 1199, was also a 
silent partner, granting a tuition loan to union members already 
employed at hospitals who became RT students. The loan will be 
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forgiven if students work for two years and three months as a union 
member in the field of radiology. 

The role of the hospitals was to provide x-ray facilities and 
related equipment and supplies for clinical experiences; in-service 
education; emergency health care services; and free lunch and parking 
for the students. 

Partner meetings took place about once a month. Instructors in the 
program had previous ties with both hospital partners, therefore 
informal contact between BCC and the partners was maximized. 

The head of the Physics Department at BCC served as the Project 
director for the RT Program and a program director and two instructors 
were hired to teach RT courses. The instructors were hired before the 
program began. The program director and instructors were experienced 
in the field of Radiologic Technology training and brought with them 
curricula and course syllabi, which were later revised. 

This program intended to serve Blacks and Hispanics, persons with 
handicaps, and older workers on a larger scale than actually occurred. 
Because the program was only 18 months long, it did not meet the 24- 
month requirement put forth by the State of New York. The State agreed 
to allow students to get credit for an 18-month program only if they 
had previous college experience. Twenty-one students were enrolled at 
the beginning of the program, however only 11 were still enrolled by 
the program's final stages. These included three white Americans and 
eight immigrants, of which three were white (Russians). 

Students participated in the program for 40 hours per week. This 
included didactic training, which took place at BCC, and clinical 
training, which took place at the hospitals two to three times a week 
from 8:30 a.m. to 4 p.m., usually by an RT instructor. Grant money was 
used in part to pay for students' tuition and a stipend. Union 
employees received an additional stipend. 

Support services were another aspect of the program. Special 
needs were identified for immigrant students having difficulty with 
English, and tutors were hired. As the program went on, students with 
academic problems received tutoring as arranged by the department. 
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The program has continued beyond the life of the grant, with a 24 
month RT associates degree program institutionalized and the continued 
involvement of all partners. Accreditation was received from the Joint 
Review Committee on Educational Programs in Radiologic Technology. 

The project was originally intended to develop a manual for use by 
institutions wanting to replicate the model, and a workshop was also 
proposed. The manual is currently in progress. Dissemination to all 
NYC/CUNY colleges took place. The dissemination of the program was 
widespread, mainly through local newspaper articles, the CUNY press, 
and in RT IMAGE, the major trade journal of the profession. 
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Center for Occupational Research and Development (CORD) 

Waco, Texas 

The Center for Occupational Research and Development (CORD), 
located in Waco, Texas, is a private, nonprofit organization that 
specializes in science and technical curriculum writing and whose 
purpose is to help educators address the technical education needs of 
workers. 

CORD used its Cooperative Demonstration Program grant to support 
the Semiconductor Manufacturing Technology (SMT) program. The goals of 
the program were to identify skills and develop a curriculum that 
satisfied the needs of the semiconductor manufacturing industry in the 
United States, to test the curriculum and skills identified by 
providing training in two pilot sites, and to disseminate the 
curriculum. Postsecondary semiconductor manufacturing technology (SMT) 
training was offered at two community colleges—Texas State Technical 
College (TSTC), which is also in Waco, and Boise State University 
(BSU), in Boise, Idaho. 

The project was administered by CORD. Its partner, SEMATECH, a 
consortium of 14 companies involved in semiconductor manufacturing, 
helped CORD to provide training by defining the knowledge and skills 
needed for those trained, as well as helping them to develop the 
curricula needed. The curricula was piloted at TSTC for the general 
student population, and at BSU for retraining of Micron Corporation 
employees. SEMATECH 's original role was as a link between CORD and 
industry to locate a retraining site. 

About six months after the grant began, the director of SEMATECH 
died. With the presence of a new SEMATECH director, there was a change 
in priorities for SEMATECH. This slowed the process of locating a 
retraining site, but worked to CORDIS advantage because CORD was then 
able to work directly with industry. Before this, SEMATECH was very 
proprietary about its industry contacts. 
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Sixtnen Micron employees completed training at BSU, and 24 
students were enrolled in the SMT program at TSTC, either as full-time 
majors or students taking SMT courses but who are actually enrolled in 
other degree programs at the college. 

Students at BSU underwent an arduous schedule, attending classes 
after work from 5:00-8:30 p.m., Monday through Thursday, for five 
months. They r?iceived 283 hours of training. Upon completion of the 
program, those trained received a SMT Pilot Program Certificate of 
Completion from BSU and program completion was noted on their internal 
Micron transcript. 

Training at TSTC included 2,400 contact hours of classroom and on- 
the-job-training. Students in the program began training in September 
of 1990 and will graduate in May 1992. The on-the-job-training 
component consisted of constructing a clean room for manufacturing 
semiconductors. Originally the program intended to manufacture diode 
packs and interpacks, however building a clean room sterile enough to 
manufacture these was unrealistic. The SMT director, who was hired 
after the grant was obtained, realized this and saw that the program 
would die without a viable on-the-job-training component. He discussed 
the dilemma with other industry professionals, who suggested 
manufacturing solar cells, which are used in calculators. The 
manufacturing process would thus remain the same as was originally 
proposed. Students were instrumental ly involved in construction of the 
clean room, an unintended but valuable benefit for semiconductor 
industry workers who frequently deal with machine breakdowns. 

The SMT Program began as a direct result of the Cooperative 
Demonstration grant. The relationship with SEMATECH was a new one, as 
was the relationship with BSU, while CORD and TSTC have had a 
longstanding and integral relationship. Several key people were hired 
to administer the program and develop the curriculum. 

The third-party evaluation of the project revealed that: 



■ The actual partnership and program develop- 
rnent model is exemplary, particularly for use 
in high technology areas; 

2 3 J 

ERIC 



■ The schedule for BSU training was unrealis- 
tic. Students were exhausted and expecta- 
tions of them were too high; and 

■ The recruitment plan at TSTI needs attention. 
The strategy is not well planned. A real 
targeted recruitment plan is needed. Some 
television advertising reached a few 
students, but most recruitment was done 
informally and from pirating other TSTI 
programs. 

The evaluator also reported that all 12 tasks as outlined in the 
proposal were met to varying degrees, and CORD is in the process of 
responding to these recommendations to ensure the future success of 
this continued program. 

Dissemination was one of the tasks proposed, and it has taken 
place on a variety of levels. National dissemination of the SMT model 
has taken place at the National Coal it ion of Advanced Technology 
Centers' (NCATC) fall and summer conferences. A conference was also 
held at TSTC in May to disseminate the model. CORD also gave 
presentations at Rensselaer Polytechnic Institute, Applied Materials 
Corporation, and published information about the model in Economic 
Development Commentary and six issues of NCATC Newsletter . The SMT 
curriculum developed by CORD will eventually be sold to any college 
that is interested. 
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Clackamas Community College 
Oregon City, Oregon 

The Cooperative Demonstration grant to Clackamas Community College 
(CCC) was used to create a model manufacturing program in precision 
metals. The program had two goals: to train entry level employees and 
to upgrade the skills of current employees in response to technological 
change. The project also targeted special groups, including unemployed 
and displaced workers, injured workers, career changers, immigrants, 
women, and other economically disadvantaged people. 

The two major partners for the project were the Oregon Precision 
Metal Fabricators Association (OPMFA) and the Northwest Screw Machine 
Products Association (NSMPA). The partnership with the OPMFA began in 
1988 when the association asked CCC to develop and conduct a training 
program specifically for the precision sheet metal industry. The 
associations agreed to loan CCC the equipment to be used in training. 
CCC then planned and offered a series of evening courses, first at the 
main campus and then at a new 6,800 square foot training facility 18 
miles south of the main campus. The Cooperative Demonstration grant 
allowed CCC to expand the curriculum, add a basic skills component, and 
target special populations of students. 

Under the grant, project staff developed three major training 
packages: a work-readiness course, a certificate program in precision 
metal fabrications, and a certificate program in screw machine 
technology. The six-week "Work Readiness" course was designed to 
provide entry level trainees with the basic skills required for 
employment in the precision manufacturing industry. As the project 
progressed, the course was renamed "Introduction to Precision 
Manufacturing Technology" and expanded to eight weeks. The course 
outline included nine areas: computation skills, basic linear measuring 
instruments, blue print reading and flat pattern development, project 
driven operation of shop machinery, industrial operation (through 
tours), communication skills, forklift operation, first aid and CPR, 
and work readiness. The forklift training was optional and added to 
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provide students with a specific skill for entry level work in many 
companies. The project paid half the cost of the $50 forklift license, 
and the student paid the other half. Because of insufficient time and 
funds, the first-aid and CPR classes were dropped but later reinstated. 

The work-re idiness program was offered six times during the grant. 
A total of 79 students entered the programs, and 65 students (82 
percent) completed training. Approximately 80 percent of the students 
were placed into jobs. Twelve students chose not to seek immediate 
employment, seven entered the nine month certificate program, two 
enrolled in unrelated college programs, one student returned to high 
school, one entered the armed services, and one (a professional 
musician) chose not to seek employment in the metals industry. The 
program was offered again in spring and summer 1991 (beyond the 
requirements of the grant). These programs lasted eight weeks, and 
each of the 18 students who attended the two sessions paid a fee of 
$1,195. All students completed the course, and five were employed as 
of October 1991. 

The certificate program in precision metal fabrication and the 
certificate program in screw machine technology were both developed by 
project staff. The project hired three instructors from private 
industry to develop the curriculum and to teach the courses. The 
programs were nine months long and allowed open enrollment at the 
beginning of each term. The maximum number of students allowed in each 
program at any one time was 15. Nine students graduated in precision 
metals by the end of the grant and seven continued into the autumn of 
1991. Eight of the nine graduating students (89 percent) are employed 
(one was under 18 and too young for employment in the industry). Nine 
students graduated from the screw machine technology program by the end 
of the grant, and nine continued into the fall 1991 program. Six 
the nine graduating students are employed at local screw machine 
companies. One student moved out of state and chose not to accept 
employment in the industry. Two other students graduated at the end of 
August 1991, one of whom started his own company. 



The metal fabrication curriculum was assembled and published in a 
book called The Shear Edge . The text was copyrighted in 1990 by the 
Oregon Advanced Technology Consortium (OATC). It contains 25 chapters 
and two appendices, one of which is a glossary. The screw machine 
curriculum also is available as are the competency based curricula for 
the work-readiness program. 

Students were recruited through a variety of efforts. Project 
staff visited local high schools, alternative schools, employment 
offices, adult and family service offices, community colleges, the 
Refugee Center of Oregon, local JTPA and PIC offices. Urban Leagues, 
dislocated-worker programs, and the Life and Career Options classes for 
abused women at CCC. The project received numerous referrals from 
private vocational rehabilitation and insurance groups seeking 
retraining for injured workers. The staff put ads in local newspapers, 
sent specialized mailings to individual industry association members 
and members of the Society of Manufacturing Engineers, and arranged for 
public service announcements on KUFO Radio. The programs and courses 
also were printed in the CCC term schedule of classes, ..hich is mailed 
quarterly to 100,000 residences and businesses. As part of the 
recruitment effort, the project produced two 10-minute videos promoting 
the job opportunities in the two areas and the two certificate 
programs. The project also produced a 30-minute repeater videotape 
containing a 75-second introduction to the OATC, which consists of CCC 
and three other community colleges that provide technology transfer and 
training to Oregon industry. 

Applicants were assessed first through a standard college 
placement exam in math, reading, and writing. If the applicant passed 
the exam, he or she met with the program instructor for a personal 
interview and performance tests involving spatial relationships. The 
personal interview allowed the instructor to adapt course content to 
the needs of the immediate group. Applicants not passing the test were 
referred to refresher math courses prior to entering the program. 
Students accepted by the program were provided free training. 



E-10 

Of the 79 students enrolled in the six sessions of the work- 
readiness program, 35 percent were female and 18 percent were minority- 
Half the students were aged 25 to 35. Of the 16 students in the 
Precision Metal Fabrication Program, 12 were male and four were female, 
and two were ethnic minorities. Of the 16 students in the Screw 
Machine Technology Program, 15 were male and one was female; none were 
ethnic minorities. In spite of advertisements in local newspapers 
targeted to minorities, there was relatively low participation in the 
project by minorities for two reasons: first, training was located at 
a facility 18 miles from the main CCC campus and not accessible by 
public transportation. Most of the unemployed and underemployed 
targeted population lived in Northeast Portland and did not have 
private transportation to reach the center. Second, the use of the 
math and reading college placement exam screened out any minorities not 
having the necessary background to score highly enough on the tests. 

The two partner associations played an active role in the project, 
and involved several other companies as well. The partners identified 
competencies needed in training, reviewed the curriculum, helped 
advertise the project, sent students to participate, donated funds to 
help with equipment maintenance and tooling costs, and arranged for 
short-term training for project staff. They also paid to print 10,000 
additional brochures and 150 videos for the project. One company owner 
and association member even served as a part-time instructor. The 
OPMFA arranged for U.S. Amada Limited to loan CCC a new $250,000 CNC 
laser cutting machine and an $80,000 bending machine. The equipment 
was loaned with the understanding that U.S. Amada could bring potential 
customers to CCC to see the equipment in operation. Other companies 
loaned other types of equipment as well or metal stock to be used in 
class exercises. 

Grant funds also were used to pay for staff development. 
Instructors for the work-readiness and certificate programs attended a 
"train the trainer" program during the summer of 1990. The three 
primary instructors all came directly from industry and had no prior 
teaching experience. They attended a "Power Presentation Skills" 
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seminar later in the year as a follow up to their earlier training. 
This seminar was designed to make instructors more effective at 
teaching adult and disadvantaged students. In addition, CCC faculty 
received training on the equipment loaned to the center. One 
instructor attended a weeklong seminar at U.S. Amada to learn the 
operation of the CNC laser cutting machine, and another support staff 
member attended for two weeks. The instructors also attended training 
in Brooklyn on the CNC plasma cutting machine. One instructor attended 
a course on the coordinate measuring machine. 

In addition to the lack of public transportation to the training 
center, the project had two other implementation problems. First, the 
downturn in the local economy after the start of the project made 
employers hesitant to hire new workers. It was difficult to place 
graduates and students had to spend more time looking for jobs than 
originally anticipated. Second, the donated machinery was expensive 
to transport to the training center and created maintenance and tooling 
needs. The original grant budget had not provided resources for the 
maintenance of equipment, and instructors had to spend time finding 
additional resources and materials. 
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Hampden County Employment and Training Consortium 
Springfield, Massachusetts 

The Hampden County Employment and Training Consortium applied for 
a Cooperative Demonstration Program grant in FY1989 after having 
received its first Cooperative Demonstration grant the year before. 
The first grant supported "Project CREATE" (Cooperative Resources to 
Enhance Access to jobs through Technology Education), which provided 
167 adults with hands-on training in the maintenance and repair of 
computerized numeric control (CNC) machinery, high tech automotive 
repair, and printing and graphics. The first grant offered eight skill 
training programs and 1,133 hours of training across all eight 
programs. The second grant supported "Project: High Tech '90," a 
program designed to recruit and train disadvantaged youth, women, 
minorities, and underskilled adults in approximately the same three 
career fields: automobile repair, graphics and printing, and machining. 
The first grant ran from January 1989 through June 1990, and the second 
grant ran from July 1990 through December 1991. 

Much of the project infrastructure and partnership arrangements 
from the first grant were carried over to the second grant. The same 
consortium staff were used on the project, although staff shifted 
positions because of the sequencing of projects. The administrative 
offices were the same, as were four of the partner organizations: 
Springfield Technical Community College, Dean Vocational Technical High 
School, Westfield Vocational Technical High School, and the 
Massachusetts Career Development Institute (MCDI). The course content 
and structure also were similar from one grant to the next, and all but 
one instructor was rehired. 

There was, however, an important difference between the two 
grants. "High Tech '90" emphasized much more heavily the recruitment 
and training of disadvantaged and nontraditional populations. Both 
project staff and partner staff were involved in recruiting students 
for the no-cost, open enrollment programs. Staff developed flyers that 
were sent to community-based organizations, prepared a videotape that 
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was played at local social services office, set up offices at the 
partner organizations, and ran cable TV and radio announcements. There 
was no screening for either prior skill levels or motivation (a fact 
later regretted), and all applicants were accepted. Project staff 
worked with the automotive instructor at Dean to translate training 
materials into Spanish for Hispanic students. Dean also hired a 
bilingual instructional aide to work with the classes. Counseling 
staff at several of the schools tracked the High Tech '90 students more 
closely than regular students. 

The project set recruitment targets of 25 percent Hispanic and 40 
percent female. The training programs were able to enroll a higher 
percentage of minorities and females than the regular programs offered 
by the partners, and several schools are continuing their minority 
recruitment after the end of the grant. The project, however, did not 
meet its target. The project was able to achieve 19 percent Hispanic, 
16 percent black, and three perce^.t Indian participation. 
Approximately 21 percent were female. One of the most difficult 
aspects of minority recruitment was getting some staff members in the 
partner organizations to change their attitude about women in 
nontraditional occupations and the involvement of minority and other 
disadvantaged populations. The partners were accustomed to being paid 
by the JTPA agency based on student performance and so were reluctant 
to take students who might not finish training or who were ethnically 
different from their other students. 

The project offered seven training programs in six different areas 
(the pretraining program was repeated twice). The first was a training 
program for 125 secondary students from nontraditional backgrounds to 
familiarize them with technology career fields. This program was 
offered twice by Springfield Technical College, and each program 
consisted of ten ours of training. Of the 125 students starting, 115 
completed the training. Because these were secondary students still 
enrolled in school, none were placed in jobs. 

The second program was 400 hours of training in desktop publishing 
and printing at MCDI. Of the 20 students participating, 13 completed 
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training, and six were placed in jobs. MCDI hired a new instructor 
from industry to teach the course because the instructor from the first 
grant was now teaching regular classes. The start of the training was 
delayed three months while MCDI searched for a new instructor. The new 
instructor then had one month to develop his course using pieces of 
courses and materials from similar courses in the area. One of the 
proiect staff served as the instructor for the desktop publishing 
component. The class applied its training to the publishing of a 
project newsletter, Access, which was printed on donated paper and 
mailed to approximately 100 businesses and organizations in the area. 

The third program was 254 hours of basic machining training at 
MCDI. Of the 20 students participating, 12 completed their training, 
and six were placed in jobs. One of the female students trained under 
the first grant became an instructor for the second grant. 

The fourth program was 254 hours of automotive repair training at 
Dean. Of the 24 students participating, 16 completed training, and 
eight were placed in jobs. 

The fifth program was 96 hours of computer aided design/computer 
aided manufacturing (CAD/CAM) training at Springfield Technical 
College. Of the 22 students participating, 18 completed the course. 
All the students were employed at the time of training, and so there 
were no further job placement activities. 

The sixth program was 48 hours training on computerized numeric 
control machining for currently employed machine shop workers. 
Training was done by Westfield Vocational Technical High School. Of 
the 15 machinists trained, all completed training and were working at 
the time of graduation. 

During the duration of the project, the economy of western 
Massachusetts was severely affected by the recession, and employment 
opportunities for new entrants into the automotive and machining fields 
became scarce. Employment opportunities in printing and graphics arts 
were somewhat better but still reduced from pre~recession levels. 
Consequently, the project had very low employment success: 25 percent 
entry for automotive, 40 percent in machining, and 55 percent in 
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graphic arts and printing. The overall impact for those employed was 
lower starting wages than had been expected, longer job searches, and 
still uncertain futures. As the recession worsened, project staff 
shifted emphasis from skill training to support services such as resume 
writing, interviewing skills, and job placement. 

There was no advisory board for the project and no general 
meetings of representatives from the partner organizations. The 
project staff's experience in the first grant showed them that 
attendance at such meetings was sporadic and that it was more efficient 
to interact with the partner organizations one-on-one. In addition, 
the consortium was also a research organization that collected job 
market information on a regular basis; no additional information was 
needed from private industry. The staff time that would have been 
spent on advisory board activities was directed toward starting 
discussion groups with high school students regarding their career 
expectations. 

The curriculum packages developed by the project (and its 
predecessor grant) are provided free to anyone requesting copies. The 
packages are suitable for use by other vocational schools and technical 
colleges. Procedures used for recruiting students may be of use to 
other schools, as evidenced by one partner school having expanding its 
regular recruiting process to incorporate the project's procedures. 

The project used the newsletter to publicize the content and 
accomplishments of the project. Although the newsletter was sent to 
over 600 individuals and organizations, including the other grantees, 
there were few requests for information. Two other grantees came to 
review the project, but it is not known if any of the project's 
features were implemented elsewhere. Copies of the CNC curriculum were 
mailed to anyone requesting them. Project staff also made presenta- 
tions at state and national conferences, e.g., AVA. 
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Home Builders' Institute 
Washington, D.C. 

The grant to Home Builders' Institute—a subsidiary of the 
National Association of Home Builders (NAHB)~~was designed to develop a 
model curriculum to train and certify employed master electricians and 
second-year apprentices as installers of SMART HOUSE. SMART HOUSE is a 
high-technology home automation energy, distribution, and control 
system, currently under development by an NAHB spin-off organization. 
The proposal anticipated instructional design and development, 
production of training materials, student recruitment, instructor 
training, pilot training at three sites, assessment, and dissemination 
to participating vocational institutions. 

As executed, the project encountered problems that led to changes 
in its scope and timing. A delay in the development of the SMART HOUSE 
technology and failure to reach agreement with SMART HOUSE developers 
on the nature of installer training led to a shift in curriculum 
development. Instead of SMART HOUSE, the curriculum that was 
ultimately developed instructs students on the installation of a 
generic home automation systervj that uses existing home wiring systei^s. 
As a result of the SMART HOUSE problems and subsequent changes, 
curriculum development was set back between 12 and 18 months. In 
addition, there was a change in project directors after the first year, 
and several changes in staff writing the instructional materials. 

Once the decision was made to develop the more generic home 
automation curriculum, information was sought from a variety of 
sources. An advisory committee was formed with members representing a 
variety of home automation manufacturers as well as users. Committee 
members reviewed curriculum drafts. Most of the curriculum writing was 
carried out by staff and consultants of Home Builders' Institute. A 
final, revised curriculum for an 18 hour installer training course was 
slated for completion in December, 1991, but the project has applied 
for an extension to June, 1992. 
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Three sites were recruited for pilot testing, which has taken 
place at two of them. The project has trained 15 instructors and 16 
students at Mid-Florida Technical Institute, and 1 instructor and 10 
students at Tidewater Community College. Students included working 
electricians as well as current students at the two schools. The 
training at Mid-Florida was conducted by a Home Builders' consultant, 
while the training at Tidewater was conducted by one of the trained 
trainers. Pilot testing at the third site was scheduled for December, 
1991, but canceled because of insufficient registration. It may be 
held in March, 1992. In addition, some instructors from pilot sites 
have received SMART HOUSE training in order to fulfill agreements made 
with the sites prior to the shift in project emphasis. 

As a result of pilot testing, the curriculum is being revised. 
Student evaluations suggested that training be made less elementary and 
more technical. In addition to changes in the course text, the project 
is developing audio/visual support (videos and overheads to introduce 
the Home Automation concept) as well as templates for laboratory hands- 
on manipulation. 

Clearly, dissemination of the curriculum has not yet occurred, as 
the curriculum is not completed. Dissemination was originally slated 
to be conducted by Partners for American Vocational Education as well 
as Home Builders' Institute. 
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Howard Community College 
Columbia, Maryland 

The grant to Howard Community College was designed to develop a 
model curriculum for a refresher course for persons returning to 
nursing after an absense of five or more years. The proposal 
anticipated changing an existing refresher course by updating the 
competencies taught and by developing and using interactive videodisc 
technology (rather than having to purchase expensive high-tech 
equipment). Howard Community College planned to work on this project 
in combination, with Essex Community College and two local hospitals 
that provide clinical sites for refresher course participants. 
Although the original proposal did not go into detail, the project 
envisioned a major revision in the refresher course already offered, 
with videodiscs developed under the grant playing a major role in 
instruction. As the proposal stated, 

A 12-week, 150-hours course will be held in the 
winter/spring semester of 1991. Eighty hours will 
be spent using the videodisc, attending class 
lectures, and in lab practice; 65 clinical hours 
will be devoted to patient/client contact, with 
another five hours in conference in participating 
hospitals. The classes will be led by the same 
instructor who had previously taught these courses 
so that significant differences in class outcomes 
can be attributed primarily to the addition of the 
videodisc rather than a change in instructors. 

As originally planned, 40 students would be recruited for the course, 
and the intended goal was a 90 percent completion rate. In addition, 
the curriculum and discs would also be "piloted" by nursing staff and 
students at a minimum of 10 institutions. 

As executed, the grant led to the development of a seven-minute 
recruitment film and three interactive videodisc "sides" (like the side 
of a tape or record) on the subject of volumetric pumps. The content 
and scripts were developed by project personnel (director and 
instructor) and reviewed by an advisory committee that included 
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hospital representatives. Actors and project personnel were used to 
tape the videos. The videodiscs, along with a workbook, were 
subcontracted to SETS, a marketing firm, and Digital Video Corporation, 
which actually programmed the discs. Project staff and the advisory 
committee reviewed the discs during development, and one of the 
interactive videodiscs was also reviewed by nursing staff at seven 
other teaching institutions. Programming for the videodiscs was still 
being refined in December 1991, six months after the grant ended, 

The videodiscs were incorporated into the refresher course for 
nurses, but delays in the development of the discs meant that students 
were exposed to a work in progress, rather than to a finished set of 
discs. Twenty-nine students were recruited to the refresher course. 
As noted by the third-party evaluator, other than the introduction of 
the videodisc, few changes were made in the refresher course as the 
competencies taught in the course were "found to be current" after 
being reviewed by project staff and the advisory committee. Estimated 
instructional time devoted to discs was about six hours. The evaluator 
found that students who used the interactive discs in instruction 
appeared to be less anxious about taking care of patients on machines, 
compared to students who took a similar course without the discs. 

One videodisc was sent to 13 institutions and seven of those 
institutions completed the feedback forms from students and faculty 
after using the videodisc. According to the third-party evaluator, "A 
total of 115 students and 12 faculty reviev/ed side one of videodisc 
one,.," and that number includes the 29 students at Howard and Essex, 

Twenty-nine students were recruited to the refresher course at 
Howard and Essex, and all but one completed the course. Not all of the 
students sought work as nurses, and because of staffing cutbacks, 
neither hospital participating in the project hired anyone who had 
completed the refresher course. One hospital had major personnel and 
organizational changes that interfered with its involvement. At the 
time of the followup, 15 of the 29 participants were employed and 10 
were in settings where high-tech equipment was being used (described as 
"nursing related work"). 
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High-technology aspects of this project included the use of 
interactive videodiscs as well as training to work on high-tech 
equipment (computer-driven volumetric pumps). The project was a new 
one for Howard Community College, which had never developed an 
interactive videodisc. The nursing program was well established 
however. The college had previously received a grant from the state to 
develop the refresher course. No new staff were hired for this 
project; all staff were already employed at Howard, Essex, or the two 
hospitals. 

Dissemination was delayed because programming for the interactive 
videodiscs needed further revision from the feedback from the pilot 
sites. The project director has presented information on the discs at 
several national meetings. She also credits the disc devi^'^opment with 
having received a Fulbright Scholarship for next year. Howard will 
sell the discs in conjunction with SETS. The project director 
indicates that the Federal monitors have okayed an arrangement in which 
SETS and Howard will share the profits, with Howard receiving 20 
percent. 




E-21 

Illinois Eastern Community College 
Olney, Illinois 

Olney Central College (OCC) is one of four branches of the 
Illinois Eastern Community College (lECC) system. OCC is a small 
college offering two-year associates degrees in a predominantly 
agricultural area of southeastern Illinois. The training in 
semiconductor technology program (SemiTop) began as a result of the 
Cooperative Demonstration grant. The purpose of the project, as stated 
in the proposal, was to train students to become technicians and 
operators in the semiconductor or chip fabrication industry and to 
establish this program as a model of business/college partnerships in 
vocational education. The objectives were to: 

■ Integrate additional technical course 
offerings at lECC into the semiconductor 
technology, level I program, which will be 
offered as part of the IFCC curriculum; 

■ Recruit at least 20 students into the program 
by August 1990; 

■ Train these students in a program that will 
lead to industrial employment or training at 
Level II; and 

■ Develop the Level II program by June 30, 
1991. 

This project intended to train students in the classroom and 
introductory laboratory experience (Level I) and develop an advanced 
laboratory curriculum (Level II). The second year of the program was 
to take place in a simulated microchip laboratory or "clean room" on 
Olney's campus, but this never happened. In the application, a diagram 
of the sterile laboratory was provided. According to the project 
director, there was an error in the type of simulated laboratory 
environment discussed in the proposal— -only an authentic sterile 
environment would make an adequate laboratory. The evaluator 
eventually discovered that there was miscommunication between the 
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project director and OCC about exactly what classroom space and 
laboratory the college would provide for the SemiTop project. The 
project director spoke of "leveraging" one grant with another and 
sought assistance from the State of Illinois to fund the sterile 
laboratory. This never occurred, and no other plans currently are 
under way to secure funds. The result was that the 19 students 
originally enrolled in SemiTop training did not receive a two-year 
degree in Semiconductor Manufacturing as originally planned, however 
they did complete the Level I courses. 

One student who did complete the class got a job with Intel 
Corporation. Eight students continued their education at OCC in a 
different program, and seven students did not complete Level I 
training. 

Level I training was taught by instructors already employed at 
OCC. A lab assistant, who also served as a full-time tutor, was hired 
as a result of the grant. Students typically took four or five 
courses — a full-time course load for the first and second semesters. 

This partnership was unique because Olney Central College's 
partner, Intel Corporation, was located hundreds of miles away in 
Phoenix, Arizona. Intel's major role was to supply equipment and 
contribute to staff development and curriculum development during four 
meetings held at OCC during the summer of 1990. Instructors received 
training which enabled them to include a semiconductor component to 
their already-existing curricula, in such areas as math, physics, 
computers, and chemistry. Intel also was going to install the 
equipment it donated as part of its role in the partnership, but this 
never came to pass. 

SemiTop dissemination included the project director's attendance 
at a SEMATECH (a consortium of 14 semiconductor manufacturing 
companies) meeting, conferences, and distribution of the curricula to 
anyone requesting it. A paper was written about the SemiTop experience 
and was delivered to the National Association for Science, Technology, 
and Society in Washington, D.C. in February of 1991 at the 
Technological Literacy Conference. A SemiTop manual was also 
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disseminated to the ERIC (Educational Resources Information Center) 
Clearinghouse. 
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Indian Hills Community College 
Ottumwa, Iowa 

The Ind i an Hills Coniniunity College's (IHCC) Demonstrcition Model 
Project was developed to provide a degree-bearing second year course of 
vocational study to non-traditional students seeking high technology 
training. The program was developed as a flexible competency-based 
program offering an opportunity for skill enhancement and a college 
degree. As stated in the proposal, the project offered three paths of 
study: 

■ A course of study in Personal Computing 
resulting in a diploma or an associate 
degree; 

■ An Associate of General Studies (AGS) degree 
in Advanced Technology, specifically tailored 
to meet student and industry need, and; 

■ Courses to be taken for skills upgrading and 
enhancement. 

The program was designed as an individualized self-paced approach to 
study, utliizing flexible study time, lab time, industry 
representatives, educational facilitators, and other college personnel. 
The project offers the non-traditional student, including women, 
minorities, displaced homemakers, and others seeking advancement 
opportunities, the chance to design a degree program made up of a 
combination of interdisciplinary courses technically-focused to meet 
real world needs. 

The project featured three educational components each being 
twelve months in length. All the programs were offered in the evening, 
and were self-paced with flexible hours and facilitators present. 

The Microcomputer Specialist Diploma component was a 24 semester 
hour offering in Personal Computing. It was a one year program for 
students seeking proficiency in microcomputer software applications and 
resulted in a diploma. This component when combined with an 
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Electronics/Computer Operations diploma from IHCC's traditional 
offerings could result in an Associate of General Studies (AGS) in 
Advanced Technology degree. 

The AGS in Advanced Technology component was designed as a second 
year degree-bearing program in high technology. The first year of 
training was obtained by students that completed the college's 
Electronic/Computer Occupations diploma program through the traditional 
curriculum prior to starting the AGS training. 

Utilizing the AGS program, with assistance from an academic 
advisor and an advisor fron the area industry, students combined High 
Technology courses and credits from other disciplines, to specifically 
gear skills to suit the needs of area industry. However, a student's 
AGS in Applied Technology plan was subject to IHCC Academic Standards 
Committee approval prior to the start of course work. 

The third component was made available to train students in second 
year High Technology programs on a course-by-course basis while 
accumulating college credit. The courses allowed current employees of 
industry to upgrade skills or train for new responsibilities that would 
enhance mobility and productivity on their jobs. 

In all three components, industry volunteer advisors were enlisted 
on as as needed basis to assist in helping students custimize their AGS 
programs. Also eight current IHCC instructors/specialists were 
utilized as "experts" to assist students in understanding high tech 
speciality concepts in each component. These experts were used four 
hours per week each to supplement the full-time educational 
facilitators during the non-traditional delivery hourr. 

In this project, students time is flexible allowing them to put in 
their contac'. hours at convenient times based on work schedules or 
child care, etc. The students must however, start and complete their 
required contact hours in accordance with the college term calender. 
It requires good time management on the part of the student and 
consistant motivation and monitoring on the part of the facilitator. 

As specified in the proposal, the program goals were: 1) to 
provide outreach efforts to 200 potential program participants; 2) 
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register 25 and graduate 15 students in the Microcomputer Specialist 
Diploma component; 3) register 15 and graduate 12 students in the AGS 
in the Advanced Technology component; 4) Register 20 and have 20 
students complete at least one course in the course-by-course 
component, and; 5) of the total number of participants, at least 66% 
3hould be women. 

The actual numbers registered were greater than expected in some 
components and less than others. However, the overall total 
registrants exceeded the original goals. Of those registered in the 
Microcomputer Specialist Diploma component, 5 completed their course 
work. The AGS in Advanced Technology component graduated 10 students, 
and the Course-By-Course component had 14 completers. 
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LTV Steel Company 
Whiting, Indiana 

The LTV Steel Company applied for a Cooperative Demonstration 
Program grant to support a portion of its apprenticeship program in 
Electronic and Instrument Repair. The apprenticeship program was 
started in 1988 by the Employee Development Department as part of LTV's 
$320 million in capital investments in its East Chicago plant. The 
installation and modification of new equipment required the creation of 
a new occupation— Electronic Instrument Repair Technician (EIRT)--and 
the retraining of a portic . of the existing workforce. An EIRT 
repairs/replaces temperature, pressure, and flow measurement instrument 
components as well as the digital and analog electrical systems with 
which they interface. 

The EIRT apprenticeship training is a four year program providing 
8,320 hours of classroom and lab instruction. Trainees are given one 
day of classes a week at the Calumet Campus of Purdue University, one 
day per week of lab at the training facility at LTV Steel, and the 
balance of time in on-the-job training. Two courses are taught each 
semester, and a semester lasts 14 weeks. The four-year program is 
considered a more desirable arrangement than having people with a full- 
time two- year associates degree plus two years on-the-job-training. 

The EIRT training is open to all of the plant's hourly employees. 
Interested workers apply to the program and must pass a reading 
comprehension and math assessment to be eligible. Apprenticeship 
positions are then awarded to eligible workers on the basis of plant 
seniority and consistent with "Consent Decree No. 1' signed in the U.S. 
District Court for the Northern District of Alabama in 1974. The 
decree specifies minority representation in apprenticeship programs in 
selected craft families. Once a worker has started in the EIRT 
program, he or she cannot withdraw for a minimum of 45 days. 
Apprentices receive wages stipulated in the current labor agreement and 
are expected to earn over $101,500 for the 8,320 hours of training. 
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The first cohort of apprentices started in June 1988 and consisted of 
25 students; 17 students still remain in the program. The second 
cohort started in September 1988 and consisted of 25 students; 16 
students remain in the program. 

LTV's grant was used to support 18 of the 48-month EIRT training 
for the third cohort of apprentices. The third cohort started in 
January 1989, and 28 students completed the grant-funded portion. The 
LTV Manager of Employee Development Programs applied for the grant 
(LTV's first federal grant application) after hearing about the 
Cooperative Demonstration Program from an official in the Illinois 
Department of Labor. The Manager had contacted the Department in an 
effort to find public funding for equipment needed to establish a 
process control lab at the Calumet Campus. LTV had received small 
grants from the state in the past to pay for various job training 
activities and LTV hoped to find similar funding. LTV agreed to donate 
approximately $150,000 of equipment for a lab if Purdue would handle 
the installation and software programming. 

The 18-month component funded by the grant consisted of three 
sections of training: 

1) Orientation (11 weeks) 

General Program Orientation and Shop Location; 
General Shop Safety 
Basic Mathematics 
Principles of Basic Physics 
Basic DC Electricity 

2) Data Collection and Communication Devices and 
Systems (37 weeks) 

Safety 

Electronic Circuits I 
Hydraul ics/Pneumatics 
Print Reading 
Basic Electrical AC 
Electrical/Instrument Devices 
Troubleshooting Techniques 

3) Control Devices and Systems (30 weeks) 
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Safety 

Electronic Circuits II 

Digital Devices and Systems 

Instrumentation 

Basic Computing and Hardware 

Instrument Test Equipment and Analyzers 

Troubleshooting Techniques 

The curricula for these courses were developed prior to the start of 
the grant by the private partner, the Electrical Engineering Technology 
Department at Purdue University-Calumet. The department assigned an 
instructor to work with LTV when LTV solicited the proposal for the 
EIRT training program in 1988. LTV had solicited similar proposals 
from other local vocational and educational institutions, but Purdue 
was the only one to respond with interest in developing curricula and 
lab experiments specific to LTV's needs. LTV selected Purdue in late 
spring of 1988 in spite of the much higher cost of their proposal. 

The Purdue instructor visited the mill several times while 
developing the initial curriculum to observe the specific control 
systems the technicians would service and maintain. Training on the 
specific equipment was critical. Purdue could train in basics of AC 
and DC circuits and equipment and fundamentals of hydraulics and 
pneumatics but didn't have laboratory facilities for process control. 
Portions of the curriculum were already available from earlier 
apprenticeship programs at LTV, but the instructor had to conduct 
extensive rewrites to update the available technology. The plant's 
Joint Apprenticeship committee (comprised of LTV Steel management 
representatives and representatives of local 1011 of the United Steel 
Workers of America) reviewed the projected summary of work processes 
and applied for Certification and Registration through the U.S. 
Department of Labor, Bureau of Apprenticeship and Training. The 
program, was certified in July of 1988. 

Students are given Knowledge Questionnaires (KQs) and 
Representative On-the-job Assignments (RPAs) to test their practical 
job knowledge. These tests and assignments are administered through 
the Electronics and Instrument Control Department and monitored by the 
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joint Apprenticeship Committee. At the time of the grant application 
there were approximately 56 KQs and 94 RPAs developed. The Purdue 
instructor often rewrites the old exams as apprentices progress into 
new units to reflect what the instructor feels is most important and 
the latest changes in technology. 

The problems encountered by LTV in implementing the EIRT 
apprenticeship program were the reverse of those encountered by 
educational institutions receiving Cooperative Demonstration grants. 
The vocational and education institutions LTV approached for curriculum 
development and instruction were unresponsive. Although the Dean of 
the department at Purdue was supportive of the partnership with LTV, 
other department faculty were not enthusiastic about teaching non- 
credit courses. The instructor who did take on the LTV assignment 
(with great enthusiasm and dedication) encountered problems in 
receiving tenure from the department because of his teaching in private 
industry. Purdue hired a new instructor to teach the third EIRT 
cohort, but LTV found the instructor unacceptable and negotiated with 
the Dean to free up additional time for the original instructor to 
handle the third cohort as well. The instructor has incorporated 
portions of the EIRT training into his regular university classes. 
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Luzerne County Community College 
Nanticoke, Pennsylvania 

Luzerne County Community College's (LCCC) Advanced Technology 
Center (ATC) Computerized Numeric Control (CNC) Cooperative 
Demonstration Training Program was developed partly as the result of a 
meeting on CNC training convened by Congressman Paul Kanjorski at the 
Wilkes-Barre Pennsylvania Chamber of Commerce* The meeting was 
attended by five regional manufacturers who said they were unable to 
operate at full capacity because of a shortage of CNC operators. 
Adding support to the need for CNC operators was a national study 
conducted by the Hudson Institute, which stated that by the year 2000 
there would be a need for 61,000 "precision production workers" in the 
United States (294 of these positions would be in the area served by 
LCCC). 

The project called for refining the existing CNC program by using 
an interactive video CNC controller simulation in conjunction with the 
College's Institute for Development Educational Activities (IDEA), 
making it suitable for those traditionally underrepresented in skilled, 
high technology operations. The revised program would target groups 
such as women, the disabled, and racial and ethnic minorities. IDEA 
would provide the client population with literacy and basic skills 
training, counseling, placement testing, and other academic support 
services required to succeed in the CNC program. The CNC training 
would be offered to qualifying students at no charge and include a $25 
per week stipend to help students with travel and lunch expenses. 
Relationships with the Luzerne County Assistance Office, the Luzerne 
County Human Resources Development Department, the local office, JTPA, 
and regional school districts were established to build a client 
referral system. 

Private sector involvement included four companies, one labor 
council, three development agencies, and three human resource agencies, 
all of which originally agreed to participate in the planning and 
operation of the project. In addition, each participating industry in 
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its letter of support and participation, indicated its strong interest 
in providing some on-the-job-training (OJT) for program participants 
and possible employment for students who completed the program. Due to 
the downturn in the economy, companies were not able to follow through 
on the previously arranged commitment, and the program could not place 
many of the students in OJT or with local industry employers. The 
major emphasis of the program therefore was focused primarily on 
student training. 

Because computerized numerical controllers (CNCs) and the 
equipment they operate are so expensive, it is not feasible for most 
training programs to hava a number of different systems available at a 
training site. To address this issue, LCCC developed and produced two 
interactive training simulation software programs, each providing one 
hour of instruction. The interactive software programs utilize 
computer-based interactive videodisc (IVD) instruction and a 
touchscreen monitor to simulate two different CNCs used by locai 
companies. With this software, the user is able to press the actual 
video images of the CNC keys (displayed on the screen) and perform 
basic simulated functions. Because the two CNCs selected for this 
project are in common usage, other training institutions arid job sites 
can use the materials LCCC developed. 

Originally, the software was to be developed in the first half of 
the project and used for training during the second half. However, the 
development of the interactive video took far more time than 
anticipated. The first disc was finished late, and the second was not 
finished by the end of the grant, so students were unable to fully 
utilize the IVD instruction. 

LCCC enrolled 58 students into the program. Twenty-six (44.8 
percent) successfully completed the program, and 3? (55.2 percent) 
dropped out. All of the students who completed the program were 
referred to employers and 10 were employed (eight of whom were hired by 
partner organizations). Six students continued with their education. 

The project modified existing courses in CNC, CIM, and math to 



E-33 

meet the lower ability level of student seeking non-traditional career 
fields. The project relied upon LCCC developmental study courses for 
remediation where necessary and added internship component for all 
students. The project used existing LCCC staff except for a project 
secretary who was hired just for the grant. 

The project staff exhibited the interactive video program at 
"Pennsylvania Technology '91" where they tried, unsuccessfully, to sell 
copies of the videodiscs. No decision had been made at time of site 
visit for further duplication and distribution of the discs. The 
project obtained newspaper coverage for the implementation of the 
interactive video. Representatives from local companies using CNC 
equipment, e.g., Midway Tools, have come to see the interactive video 
training for possible use in their company. Copies of the curriculum 
packages were sent to ERIC. 
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Nebraska Department of Labor 
Lincoln, Nebraska 



Job Training of Greater Nebraska (JTGN) is an agency of the 
Nebraska Department of Labor that operates employment and training 
programs under the Job Training Partnership Act. The Rural Allied 
Medical Business Occupations Program (RAMBO), funded by the Cooperative 
Demonstration grant, trained respiratory therapists, drug and alcohol 
counselors, and licensed practical nurses (LPNs). Also one student was 
trained in histology and one as a lab assistant. 

The major goals of this program were to establish partnerships, 
increase access to health care training, and create new or improved 
training. But the program's ultimate focus became one of finding jobs 
for needy people. The five objectives, as outlined in the proposal, 
were to: 



■ Help solve rural Nebraska hospitals Allied 
Medical shortages based upon needs; 

■ Provide economically disadvantaged 
individuals. ..with high-tech medical training 
which will lead to professional standing in 
rural Nebraska communities; 

■ Furnish Job Service, Department of Social 
Services, Department of Education, and Job 
Training of Greater Nebraska with the 
opportunity to refer disadvantaged 
individuals for training in high-tech 
occupations. 

■ Provide individual quality training in a 
high-tech field over a 12- to 18~month 
period. 

■ Provide on-the-job training/classroom 
training opportunities in high-tech medical 
disciplines at Saint Francis Medical Center 
and Central Community College in Grand 
Island. 
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JTGN, in Lincoln, Nebraska, teamed with Saint Francis Medical 
Center and Central Community College, both in Grand Island, Nebraska. 
The JTGN role was to recruit students and coordinate the partnership. 
Saint Francis' role was to provide on-the-job-training, and Central 
Community College's role was to provide classroo:;! training. The 
partnership planning began one year prior to the grant, but the 
partnership itself began with the grant. 

This partnership was distinct in that it consisted of a very 
tight-knit relationship among three dedicated partners. The director 
of RAMBO at JTGN, Central Community College's institutional advancement 
director, and Saint Francis Medical Center's director of human 
services, were dedicated to RAMBO and put for+.h many hours on their own 
time to ensure the students success. 

Support services were also a unique aspect of this program. 
Remedial help in the form of supplemental coursework taught at CCC was 
provided to students who needed it. Financial support was also given 
to students, many of whom were on welfare, to pay for tuition, to help 
them relocate to Grand Island, and to purchase necessities such as 
eyeglasses and suitable clothes for job interviews. 

All training programs were one year, and all included on-the-job 
and classroom training. The LPN program included 500 clinical hours. 
Drug and alcohol training included a 30-day in-patient therapy 
component; the clinical component included observing group therapy 
sessions, conducting counseling sessions, and learning how to write 
master treatment plans. Trainees also were given a caseload for 
several months at the end of their training, under the observation of 
program staff. 

Some problems with RAMBO included student retention. Twelve 
students in the program dropped out for a variety of reasons, ranging 
from academic difficulty to problems with self-esteem. RAMBO staff 
continually urged these students to stay in the program. Eventually 26 
students finished the RAMBO program, all but two of whom were hired 
following graduation. Some students also had problems regarding State 
certification requirements for drug and alcohol counselor and certified 
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respiratory therapist licensing, although petitions of waiver for the 
State requirements were submitted and granted. 

The partners are trying to secure State funding with the help of 
the lieutenant governor in order to continue the RAMBO program. The 
plan is for the Department of Social Services to assume students' 
social service needs, JTGN to cover students' tuition needs, and for 
the State to pay for a project administrator. If the program 
continues, the surgical technology and certified respiratory therapy 
programs will not be included because of insufficent need for surgical 
technologists as well as the AMA requirement that certified respiratory 
therapists be observed over a three-month period, a requirement that 
was waived for persons trained under this grant. 

No dissemination activities were proposed, although substantial 
dissemination occurred through television spots, newspaper articles, 
presentations delivered at national conferences, as well as articles in 
professional journals. All three partners gave presentations at the 
following conferences: American Society for Healthcare Human Resources 
Conference, the Adult Learner Conference, and the National Conference 
on Rural Adult Education Initiatives, 
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North Clackamas School District #12 
Milwaukee, Oregon 

The grant to North Clackamas School District was used to establish 
the Student Cooperative Training Units (CTU) Program at the district's 
Owen Sabin Occupational Skills Center. The district had been 
approached by the Northwest Regional Educational Laboratory (NWREL) to 
determine if the district would be interested in applying for a 
Cooperative Demonstration Program grant. NWREL helped the district 
prepare its proposal and continued to take an active role in the 
project as both a technical support and third-party evaluator. Other 
partners in the project were Precision Castparts Corporation, Block 
Graphics Inc., and Providence Milwaukee Hospital. These partner 
organizations already had working relationships with the Center's 
instructors, and those relationships were used to create formal student 
internship positions at each company. 

The grant had seven objectives: 

■ To coordinate the development of curriculum 
for Advanced Information Systems, Graphics 
Technology, and Health Occupations 
occupational cluster programs to prepare 
students for entry into those fields; 

■ To provide students with practical training 
using high technology tools; 

■ To pilot the CTU Program in each of three 
occupational areas: Office Systems, Printing, 
and Health Careers; 

■ To refine and revise the curriculum as 
necessary; 

• To operate a full school year CTU program; 

■ To evaluate the project's impact on 
participating staff, students, aid business; 
and 
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■ To disseminate the project's approaches, 
processes, materials, techniques, and 
findings to state and national audiences. 

The Advanced Information Systems (AIS) cluster trained 
approximately 40 students per year in office systems using a networked 
computer terminal and a variety of word processing, electronic mail, 
scheduling, and information management software. Students enrolled in 
the second year of AIS were eligible to apply for the CTU internship at 
Precision Castparts Corporation. 

The Graphics Technology cluster trained approximately 40 students 
per year in desktop publishing, comprehensive layout, paste-up, 
masking, platemaking, offset press operation, hazardous chemical 
handling, and safety. Students enrolled in the second year of the 
cluster were eligible to apply for the CTU internship at Block 
Graphics, Inc. 

The Health Occupations cluster trained approximately 140 students 
per year in human anatomy, medical terminology, and health-related 
skills. Students enrolled in the second year of the cluster were 
eligible to apply for the CTU internship at Providence Milwaukee 
Hospital . 

Teachers selected CTU participants based on a student's maturity 
level, behavior, academic performance, and career interest. Each 
candidate was referred to a business partner, and students were 
required to follow the hiring process normally utilized by the business 
partner's company. They were interviewed by the personnel officer and 
then referred to the appropriate site supervisor for additional 
interviewing or skills testing. Students accepted as interns received 
new employee orientation, a tour of the company, and an employee 
handbook to review. Upon graduation, students were expected to enter 
the work force in that industry or continue their education at the 
community college or university. The private partner made no guarantee 
to hire students after the internship ended. Although both male and 
female students participated in the project, no minorities were 
trained. 
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The CTU program was operated in two phases: a summer pilot phase 
and a school year phase. The summer pilot phase began with an 
orientation session for interested students. Students indicated they 
would be interested in doing an internship if high school credit was 
given and if they were paid a stipend to replace the money they 
otherwise would have earned through summer jobs. Each student was 
authorized to receive up to 180 hours of training over a four- to six- 
week period. During the pilot phase it became clear that it was not 
feasible tc train students in groups as had been originally proposed. 
To compensate for the lack of interaction between students on site, 
interns were required to record daily entries in journals (in order to 
improve their communication skills), and attend debriefing sessions 
where they discussed their experiences. Instructors reviewed the 
journals each week. 

During the summer pilot phase, 12 AIS students started 
internships, and all completed their training. Of the 12, six were 
hired the following year under a cooperative employment agreement (even 
though the company had a hiring freeze), five continued their 
education, and one was still looking for a job. Seven Graphics 
students started internships, and all completed their training. Of the 
seven, two were hired the following year, three continued their 
education, and one was looking for employment. Five Health Occupation 
students started internships; four completed training (the fifth 
dropped out because of a schedule conflict with a second job). None of 
the five could be hired by the partner because Oregon State regulations 
required postsecondary training to qualify as entry level technicians 
in the healthcare industry. One of the most substantial contributions 
the summer interns made was to draft, edit, and refine a new 
instruction manual for a software packaoe (Occupational Health 
Maintenance) purchased by Precision Castparts. 

The school-year phase of the project included continued 
internships at the three partner companies. However, due to school 
classes and other student activities, students were not able iu train 
for six hours per day as they had done during the summer. The training 
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schedule was first modified to allow three hours per day on-site, but 
companies had difficulty with that schedule and asked that the time be 
increased to four or more hours per day. As in the pilot phase, 
students were placed in internship positions for a period of four to 
six weeks. During the school year, six AIS students, eight Graphics 
students, and 11 Health Occupations students started internships, and 
all completed their training. 

The five teachers at the center who taught the three cluster areas 
scheduled weekly conferences with the internship supervisors at each 
company, during which the teacher reviewed each student's performance, 
learned about the impact of technology on the workplace, and became 
familiar with new equipment or industry procedures. The teachers 
integrated their new knowledge into their program curriculum and 
customized their courses to local industry needs. Teachers needed 
continually to revise their curriculum to fit the rapid changes in the 
industry. For example, the AIS cluster put more emphasis on personal 
computers, WordPerfect, and Lotus 1-2-3, 

Representatives from the three partners and other businesses sat 
on the advisory committee for the Skills Center (there was no separate 
grant advisory committee). The committee was asked to review revisions 
to the curriculum, and throughout the year teachers discussed with 
individual committee members any special problems they might be 
experiencing. Through the committee and through weekly conferences, 
the partners made specific suggestions about what topics they wanted to 
see included. Precision Castparts requested that excepts from their 
employee handbook regarding grooming, attendance, and performance 
expectations be included in program orientation. Block Graphics asked 
that training in handling hazardous chemicals become a prerequisite for 
placement at its facility. Providence Hospital suggested that more 
emphasis be place on computer literacy. 
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Partners for American Vocational Education (PAVE) 
Alexandria, Virginia 



Partners for American Vocational Education (PAVE) is a private, 
nonprofit foundation organized to create partnerships between business, 
industry, and education, PAVE managed the Cooperative Demonstration 
grant, which consisted of partnerships established between a coalition 
of employers and a coalition of educational institutions, PAVE is in 
Alexandria, Virginia, and most of the employers and educational 
institutions involved in this grant are located in nearby Washington, 
D,C, 

This project, called Business-Education Venture for Health Care 
Occupations Training, had two primary goals: 

■ To develop and implement an effective Busi- 
ness-Education Venture that will maximize the 
resources of education institutions and 
health care providers in the District of 
Columbia to train and place skilled tech- 
nicians in the health care industry; and 

■ To field test a health care high technology 
Business-Education Venture that will enroll 
150 persons in training programs that will 
provide them with skills to increase their 
value, performance, and employabil ity as 
technicians in the health care industry. 

Student training was conducted in five areas for health care workers: 



■ Nurse occupations technologies, which 
provides a minimum of 75 hours of lab work 
and on-the-job training: 

■ Medical transcription technology; 

■ Phlebotomy technology; 

■ Medical records apprenticeship; and 

■ Medical unit clerk courses. 
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These training programs typically were six to nine hours per week 
for 12 to 14 weeks. Cost per student ranged from $25 to $150. 
Participants registered by telephone through the PAVE-run Health Care 
Training Hotline. Prior to training, students were assessed using 
Valpar MESA Short Form, an assessment software package. 

Approximately 71 students, half of whom were already employed, 
completed specialized health care training provided by PAVE. 
Approximately 75 percent of those students not having jobs upon 
entering the programs found employment upon completing the program: 

■ One hundred percent of 18 phlebotomy 
technology completers; 

■ Seventy-one percent of 35 nurse occupations 
technologies completers; and 

■ Seventy-two percent of 18 medical 
transcription technologies completers. 

The medical records apprenticeship and medical unit clerk courses 
did not yield certificates of completion; 27 students took part in 
these courses. Fifteen students received some degree of remediation, 
which PAVE managed using BASF sof -^re. 

One unforeseen problem PAVE experienced was the extensive "fall 
out" between the time initial contact was made and participants 
actually were enrolled in training: although 255 students initially 
contacted PAVE, only 196 enrolled in training. And, a high number of 
students (35 percent) dropped out of programs before their training was 
completed. 

PAVE's relationship with its partners was different from other 
Cooperative Demonstration grantees for several reasons. First, PAVE's 
role was to manage partnerships. It oversaw the partnerships formed 
between the coalitions of health care employers and education 
institutions mentioned earlier. The coalition of employers included 
the Washington Hospital Center, Howard University Hospital, Greater 
Southeast Hospital, Childrens' Hospital, Little Sisters of the Poor (a 
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long-term care nursing home), and the Washington Nursing Facility, The 
educational institutions that provided training included the District 
of Columbia Public Schools and the University of the District of 
Columbia. Thus, PAVE established many different networking 
partnerships with each of these institutions, and it facilitated 
training between them. This third-party management feature made this 
program unique. 

This grant also was unique in that PAVE took over the role of 
referring students to existing training programs, although some of 
these programs were adapted to meet the needs of PAVE trainees. PAVE's 
role was to recruit students for existing training, instead of the more 
common Cooperative Demonstration grantee role of actually providing the 
training. 

High-technology aspects of this program were the nursing 
procedures and technologies used, particularly in the phlebotomy 
technology and nurse occupations technologies training programs. 

Dissemination of program activities were to include a guide on the 
development of a Business-Education Venture, a manual to facilitate 
model replication as outlined in the proposal. Instead, a briefer 
version of this manual, with a broader focus, was developed in the form 
of guidelines. These were disseminated to State directors of 
vocational education and State councils on vocational education. 
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Southwestern Community College 
Chula Vista, CA 

Southwestern Community College (SWCC) received a Cooperative 
Demonstration grant to design and implement a Comprehensive Aerospace 
Manufacturing Technology Program. As originally proposed, the college 
was to develop a program that would train disadvantaged individuals for 
jobs in aerospace; upgrade skills of current aerospace employees to 
make them more promotable; and bring together high schools, four-year 
colleges (San Diego State University), and companies (especially Rohr 
Industries) through Southwestern Community College. The project 
planned to serve three groups: local high school students, existing 
SWCC students and persons not employed in the aerospace industry, and 
industry upgrade trainees (employees). There were four major program 
components: recruitment, assessment and placement in appropriate 
educational /training program, training (basic educational skills, core 
aerospace skills, and advanced technical skills), and student/trainee 
support services and job placement. 

As delivered, this project appeared to provide some services to a 
number of different groups, but there was little relationship mong the 
services. Each of the three target groups received some service. It 
is hard to determine which services the project initiated from scratch 
and which were added to services, such as mentoring, tutoring, and 
counseling, already offered at SWCC. The thread that appeared to run 
through the overall effort was an attempt to reform instruction (or 
build capacity) at SWCC. 

The downturn in defense procurements wiped out the growing need 
for aerospace employees at all levels. Rohr Industries, slated to be 
the primary partner, laid off one-third of its employees. The Rohr 
employee who was to be the full-time industry liaison for the project 
lost his job. In addition, the project encountered problems in 
staffing. The proposed project coordinator could not wait for an 
uncertain grant. It took five months to find a new coordinator. 
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Despite these problems, the project did accomplish several 
objectives. It carried out tutoring, curriculum development, revis 
of the engineering technician and engineering transfer programs, 
developed workshops, placed interns, and linked the institution to 
NovaNet, a system that allows students access to thousands of 
interactive, educational software packages, many of which teach 
remedial math and science. The project coordinator was able to 
establish limited partnerships with other companies. 



i Tutoring. The project supported after-school 
tutors in high school math and science during 
the 1990-91 school year. The project esti- 
mates that 250 students received some 
tutoring. 

■ Mentoring. Forty-one SWCC and 17 high school 
students were contacted by and/or partici- 
pated in a mentoring program sponsored by the 
campus' chapter of the Society of Hispanic 
Professional Engineers (SHPE). 

■ Competency-Based Instruction. Ninety-eight 
SWCC students participated in classes in 
which instructors had received stipends for 
curriculum development aimed at making the 
courses competency based. 

■ Career Counseling. Twenty-six SWCC students 
attended job placement workshops, 16 produced 
resumes, 30 attended a Careers in Engineering 
workshop, 87 attended a career counseling 
overview session, and 44 obtained one-on-one 
counseling session at the Career Center on 
campus. 

■ Remediation. The project installed five 
terminals on the NovaNet system. Approxi- 
mately 100 students used the remedial 
software. A number of SWCC students improved 
their scores on an algebra readiness test 
after working on the NovaNet remediation 
system. 

■ Internships. Despite the downturn in the 
local aerospace industry and the general 
economy, the project coordinator was able to 
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place several students in internships at 
local companies. Six summer positions were 
developed in 1991. There are currently 11 
industry requests for 1992. 

i Classes at Rohr. Five SWCC classes were held 
at Rohr for Rohr employees. A total of 87 
employees participated. In addition, 55 Rohr 
employees received some counseling, and 14 
used NovaNet. 



As for the high-technology focus, the project planned to train 
students and workers with varying skill levels to work in the aerospace 
industry. Courses already available at SWCC dealt with manufacturing 
engineering, computer-aided manufacturing, industrial engineering, and 
quality engineering. 

Given the extremely local nature of the services and the 
difficulties this project encountered, it is not clear what elements 
are exportable to other sites. Some of the curricula developed at SWCC 
may be usable by others, and other institutions may want to look into 
SWCC's experience with the NovaNet remediation system. 
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State Center Community College District 
Fresno, California 



State Center Community College received a Cooperative 
Demonstration grant of $399,000 to establish an Advanced Technology 
Center and provide high technology manufacturing training to 
disadvantaged populations. The proposal envisioned the development of 
an Advanced Technology Center (ATC), complete with new facilities and 
curricula, located in downtown Fresno. The ATC would recruit local 
unemployed, disadvantaged, and refugee populations through social 
service agencies and train them in high-tech skills needed by area 
manufacturers. Courses were to cover topics such as Industrial 
Controls, Hazardous Materials, Management of Manufacturing Processes, 
Just In Time (JIT) Inventory Controls, Computer Aided Design (CAD) and 
Computer Aided Manufacturing (CAM) and NC Programming. Courses were to 
be taught in short but intense modules (9 weeks, 20 hours per week). 
After completing the courses, trainees were to be placed in three-month 
internships at companies represented on the project advisory board. 
The project would create a Manufacturing Technician Certificate Program 
and establish an articulation agreement with California State 
University at Fresno. After the project, the ATC would serve primarily 
as a short-term technical training center for the employees of local 
manufacturers. 

As delivered, the project established an ATC and developed 
curricula in Hazardous Materials, Computer Assisted Design (CAD), 
Computer Automated Manufacturing (CAM), and industrial electronics 
(including the operation of programmable logic controllers). The ATC 
is located on the campus of Fresno Community Co*;iege, rather than 
downtown, because the promised warehouse space never materialized. 
Furthermore, there has been little training of unemployed or 
disadvantaged populations. According to project staff, representatives 
of companies on the Advisory Board insisted on levels of training 
beyond what could be handled by most disadvantaged students. Most of 
the training went to employees of local manufacturers. The ATC (both 
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the facility and the curricula) were incorporated into the college's 
Industrial-Tech. Department at project end. 

This project had several accomplishments: it acquired a wide range 
of high tech equip^ient for training at the ATC, it gave instructors 
support to develop and deliver the ATC curricula, and it strengthened 
the college's relationships with local manufacturers by allowing them 
to play a role in curriculum development and by training their 
employees (with grant funds). 

Much of the training provided by the project was equipment- or 
software-specific. For example, students/employees took a class to 
learn how to operate a particular piece of equipment, such as a 
programmable logic controller (PLC). The original design of the 
classes as nine-week modules required this high level of specificity. 
In several classes, however, this nine-week design was changed. In the 
Hazardous Materials classes, the nine-week time period was simply too 
short to learn about all the different materials and regulations. In 
reality, the classes became more like normal college classes — 3 hours 
a week for 18 weeks (one semester). According to project staff, the 
scheduling of the classes was often determined by the company 
representatives on the advisory boards, and was designed to match up 
with the workshifts at local companies. For instance, PLC classes were 
taught in the early morning so that employees could attend before work. 

In several of the classes, such as PLC and CAD, instructors 
discovered mid-way through a course that a number of students/employees 
lacked the basic math and electronic skills necessary to complete the 
class. The instructors had to stop teaching the specific skill 
(equipment or software) and teach the basics. (Several industrial 
electricians did not know what ohms and amperes measured, and several 
drafting students were not proficient in trigonometry.) 

As part of the incorporation of the ATC into the college's 
Industrial-Tech department, the college is working to develop a broader 
two-year certificate program in Advanced Technology. All students in 
this program will take core classes in math and electronics, then 
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select a specialty in Hazardous Materials, industrial electronics, or 
CAD/CAM- 

A total of 124 persons received training. Of these, 98 were 
current employees of companies. A wide variety of courses were 
offered, including CNC control (first class 20 hours, second class 4S 
hours~24 students registered and 22 completed), Basic Programmable 
Mill (first class — 20 hours^ second class 45 hours — 18 students 
registered and completed training), CNC Manual Programming (1 class, 8 
students, 90 hours of training). Tool Design (1 class, 8 students, 90 
hours of training), Basic Programmable Lathe (7 students, 45 hours), 
CNC Programming (9 students, 45 hours of training), Intro, to Hazardous 
Materials (2 courses, 54 hours each, 14 students each), and Industrial 
Hazardous Waste Treatment (2 courses, 54 hours each, a total of 16 
students) . 

As for exportabil ity of what was learned in this project to other 
sites, the original proposal noted that as of February 1989, 41 ATCs 
had been established in 18 states. The project director pointed out 
that most these ATCs are in hard-hit industrial areas, Fresno is 
largely an agricultural area, where the potential users of high- 
technology equipment are in food-related industries such as processing 
and packaging plants. The establishment of the ATC is also part of an 
effort to attract non-agricultural industry to Fresno, 
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Waubonsee Community College 
Sugar Grove, Illinois 

The office technology program, "Survival Skills for Office 
Technicians", has been operating at Waubonsee Community College (WCC) 
since the late 1970s and has been supported by a variety of public 
funding sources. The program has been distinct from the regular, for- 
credit office careers training at WCC both in sources of funding and 
philosophy of operation. Where postsecondary students would pay to 
enroll in office careers classes and receive instruction in a 
traditional classroom structure, students in the office technology 
program had their training costs paid by another agency and worked at 
their own pace. The program also was distinct from the customized 
training offered to local employers. Prior to WCC's first Cooperative 
Demonstration grant in FY1988, most of the noncredit office technology 
training was supported by CETA and. Job Training Partnership Act (JTPA) 
funds through the Kane-DeKalb County Private Industry Council (KDC- 
PIC). 

WCC used its first Cooperative Demonstration grant to provide 
basic skills assessment, training, and job placement to 208 minority or 
disadvantageo women. The first grant provided training in basic 
keyboarding, filing, bookkeeping, switchboard operation, office 
decorum, database management, word processing, and electronic 
spreadsheets. The second grant (FY1989) added training in Unix-based 
software (SAMNA and Lotus), electronic mail, electronic shorthand, fax, 
and desktop publishing. The second grant also enabled the instructors 
to place more emphasis or computers. For example, the typing course 
went from using manual typewriters to computers, and the filing course 
went from using a manual file box to a computerized filing system. 

Under both grants, classes were held at WCC's downtown Aurora 
campus from 8:30 a.m. to 4:00 p.m. Monday through Friday (with two 15- 
minute breaks) to simulate the actual office schedules. Evening hours 
were added during the first grant period to accommodate those people 
already working, but students had to commit to at least 12 hours of 



E-51 

instruction per week. The program is continuing in its entirety after 
the end of the grant. 

Although the "Survival Skills for Office Technicians" program is 
an outgrowth of an existing program for JTPA clients, students are 
eligible for training only if they are not eligible for JTPA. The 
exclusion of JTPA clients is in part to satisfy concerns of the KDC-PIC 
about competing for clients, but also in part specifically to serve 
those women not being served by JTPA. For example, WCC was aware of 
widows and divorcee:; who would have had to sell their houses in order 
to be eligible for JTPA. The grant was designed to assist these women 
as well as others with no family income or assets. Child care was 
provided on campus and paid for by the project if necessary. 
Approximately 94 percent of the people trained under the second grant 
were women and 49 percent were minorities; 24 percent were unemployed 
at the time of training. 

Students were referred to the program through private employment 
agencies, partner organizations, and the WCC counseling office. 
Project staff advertised the program and classes through church groups, 
the Urban League, the local Migrant Council, and other community-based 
organizations; staff were approached by a prison-release program to 
include those clients, as well. Students admitted to the program had 
to have a minimum of a iOth grade reading/comprehension level and a GED 
or high school diploma or be in a GED class. Basic skills levels were 
assessed through ASSET tests given by the college's assessment center. 
There also was a two-day assessment in the Office Technology Department 
for pretests in math, English, spelling, filing, and typing. 

The partners in the grant were companies that had either referred 
individuals to the program in the past or who had hired people trained 
by the program. For example, one of the 12 partners was the Illinois 
Department of Employment Security (IDES), which referred people filing 
for unemployment to WCC for retraining. The project director would 
certify to the IDES those students being trained each week so the 
students could continue to collect benefits. Another partner was the 
Kane County Circuit Court which had hired students from the program in 
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the past. A third partner was the Kane County Department of Public 
Aid, which both referred people to the program under its own "Project 
Chance" and which hired graduates of the program. There were no formal 
partnership agreements between WCC and the private groups, and the 
project had no formal advisory board of partners. 

The program included an unpaid work-experience component for those 
students who could type 30 words per minute and who had some word 
processing skills. Approximately 15 percent to 20 percent of the 
students were placed in internships at any one time. The project 
director arranged for placements in the partner organizations. 

All of the seven project staff members were part-time (35 
hours/week) employees of WCC and all were paid entirely from grant 
funds. As part-time employees, staff received no fringe benefits other 
than retirement. The lack of benefits made it difficult for the 
project to attract and hire staff. This lack of benefits made staff 
members even more aware of the importance of placing students in jobs 
with benefits. Consequently, the staff made an extra effort to find 
students jobs with public agencies. 

Of the 40 courses offered by the program, 15 were new courses 
during the second Cooperative Demonstration grant. The new courses 
were on subjects either suggested by partner organizations (e.g., 
customer service) or by student feedback on their work experience 
(e.g., proofreading). The new courses were developed by project staff 
using commercially available products (e.g., a proofreading curriculum 
package) or from portions of software manuals and supplementary texts. 
The office telecommunications course was developed by an intern from 
Indiana University, who was paid only for her teaching time but who 
received college credit for developing the course. The staff purposely 
did not look at courses from the other colleges because the staff felt 
that the students being trained needed an approach different from that 
used in for-credit courses. 

The open-entrance\open-exit and self-paced structure for the 
training are available for export but are not well documented. Copies 
of curriculum packages are available to other institutions upon 
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request. Northern Illinois University has implemented the packages for 
WordPerfect, keyboarding and data entry, and customer service. Two 
local school districts are reviewing the packages for possible use. 

The project director has arranged for three articles about the 
project in the local newspaper and appeared on two TV shows. The 
newsletter for all Cooperative Demonstration grantees that was started 
under the first grant was discontinued after four issues in the second 
grant. 
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West Virginia Northern Community College 
Wheeling, West Virginia 

West Virginia Northern Community College (WVNCC) is a two-year 
college located in Wheeling, West Virginia, with three branch campuses, 
including one in Weirton, Weirton, in the heart of steel and coal 
country, is the home of Weirton Steel Corporation, the seventh largest 
steel company and the only employee-owned steel firm in the United 
States. Twenty percent of all Weirton residents work in the steel 
mill. 

The partnership goals were to develop a curriculum and provide 
upgraded training for current plant workers. The training included 
high-technology computer education for workers that would enable them 
to operate a new da':a collection system at the plant. Training also 
would include craft instruction in high-tech skills for pipefitters and 
millwrights so they could operate now, automated equipment for hot 
strip mill renovation at Weirton. 

The new data collection system that became the focus of the 
training was the Integrated Mill Information System (IMIS), purchased 
from Computer Services Corporation. IMIS was purchased to better track 
products being manufactured in the mill and more quickly ascertain 
their production status in order to enhance customer service. Due to 
problems in implementing IMIS, plans changed from training plant 
workers to conduct statistical and other advanced analyses to training 
them in data entry and initial computer orientation. Although the 
training program was essential for workers to learn IMIS, similar 
training with newly implemented information systems typically is 
conducted by the vendor. 

Classroom training took place at three-hour sessions for three 
days at Weirton Steel's classroom facilities, after regular work hours. 
One crew (per mill operating unit) was trained at a time. Employees 
received on-the-job-training for about two weeks, although more was 
available if needed. Weirton Steel compensated all 1,200 employees who 
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received IMIS training in the form of five hours of vacation time for 
every three hours of classroom training, as well as free meal tickets. 

Curriculum development was a major undertaking but an essential 
part of the work of this program. Forty-four curriculum modules were 
developed for IMIS training, one module for each machine at Weirton 
Steel, Staff development also was extensive. Trainers were carefully 
screened and, when hired, trained for 40 hours a week for six weeks. 

The craft training component of the grant was a much smaller 
dimension than originally specified in the proposal. Workers in the 
hot mill were vendor-trained, with WVNCC trainers playing only a 
minimal role. Craft training was given to 389 hot mill workers. 
Topics of craft instruction were hydraulics, lubrication, shaft 
alignment, welding, pipefitting, and scaffolding. Classes were offered 
for eight-hour sessions during the work week. To complete the 200-hour 
craft training program, employees attended class 40 hours per week for 
five consecutive weeks. 

The relationship between WVNCC and Weirton Steel was not a new 
one. They had already formed a partnership in 1987 for the Workplace 
2000 Workplace Literacy Program, This training program was a new one, 
however, and in order to run it seven full-time and five part-time 
trainers were hired. Also user support team and plant floor support 
team members— all Weirton Steel employees--facilitated on-the-job use 
of classroom learning. 

Dissemination activities included a national teleconference called 
"Education: Bridging the Gap," which included topics such as the 
business/industry partnership, employees' fear of change, and assessing 
training effectiveness, WVNCC staff made presentations in Florida— at 
the Leadership in Education conference, Florida State University's 
national conference, and in California— at the International Conference 
for the Council for Adult and Experiential Learning, 
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Fox Valley Technical College 
Appleton, Wisconsin 



Fox Valley Technical College (FVTC), located in Appleton, 
Wisconsin, offers 59 associate degree and vocational programs to more 
than 5,000 students, as well as contracted training to approximately 
17,000 local industry employees per year^ Appleton is in a part of 
Wisconsin that is highly oriented to manufacturing. 

The goals of the Computer- Integrated Manufacturing Program (CIM) 
were to demonstrate a cooperative approach to provide training and 
technical assistance in CIM to local manufacturers and develop a model 
that could be used by other postseconoary educational institutions. 

The specific goals of the CIM program, as stated in the proposal, 
were to provide: 1) Orientation to CIM for small businesses through a 
college-wide CIMulation Laboratory; 2) Cost-effective training in CIM 
via a quality approach to instruction; and 3) Comprehensive, yet 
affordable, support in CIM planning and implementation from an expert, 
faculty-student technical assistance team. 

The first steps needed to carry out these goals were to increase 
the client base and market CIM. To that end, individual instructors 
reached out to industry. Also, the CIM Solution Demonstration (CSD), a 
storybook demonstration on computer, referred to as the CIMulation 
laboratory in the proposal, was instrumental in promoting marketing. 
It was put into place with Cooperative Demonstration grant funds. 

The CSD shows how a small manufacturing company uses the 
technologies of an integrated system to improve its management, 
operation, and responsiveness. The live demonstration tracks an 
engineering change from a request for quotation to the shop floor where 
a prototype part will be cut to meet a customer's specifications. All 
departments — such as management, operations, and production ~ 
explain their role in this engineering change, and the demonstration 
concluded with a final quotation being printed for the customer. 
Conversion to CIM technology for this company resulted in improved 
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response time and an improved market share. Every Thursday, businesses 
interested in CIM were given two-hour tours through the CSD. To 
increase local business exposure to the CIM Solution Demonstration, a 
student phoned local businesses to invite them to the demonstration, a 
practice which continues. FVTC used grant money to supplement the cost 
of training as an incentive to get companies to participate in CIM. 
CIM training usually was held on the campus of FVTC at the Sordini 
Center, which was especially built to provide corporate training and is 
a symbol of FVTC's commitment to industry. This arrangement was 
preferred to on-site training because there would be fewer 
Interruptions, but in some cases on-site training was essential because 
specialized equipment was only available there. Once CIM was 
Implemented on-site, a CIM instructor would spend time at the company 
making sure implementation was complete. 

Not only was the CIM curriculum offered to companies, it was also 
integrated into other FVTC degree programs, such as accounting, data 
processing, and printing and publishing. Approximately 1,300 students 
at FVTC completed a course that was infused with a CIM curriculum 
component, and 216 persons already employed by business and industry 
completed training. 

Prior to the grant. Fox Valley entered into a relationship with 
IBM, its partner, because the college was chosen by IBM to become a 
member of the CIM Alliance. The CIM Alliance is an alliance of 70 
colleges and universities throughout the United States that are active 
in CIM technology. The CIM Alliance allowed Fox Valley to exchange 
technology and develop CIM curriculum. IBM donated computers and 
computer programs. 

There was a small change in strategy from what was proposed in the 
application to what actually happened with CIM training. One goal of 
the project was to put together a team of instructors to go out into 
industry and offer a CIM system that effected the whole company. This 
has not worked well, however, because area companies would rather 
undergo piecemeal change rather than company-wide change. Therefore, 
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CIM training more typically takes place using one particular 
application, such as CAD to CAM, using AutoCAD software. By taking on 
these small CIM application training sessions, CIM instructors hope to 
spread the word about CIM availability and the possibility of company- 
wide change. 

The third-party evaluation was particularly valuable to the CIM 
staff, causing them to institute some changes that worked to the 
benefit of the program. The strengths of the FVTC program identified 
by the evaluator included: 1) FVTC is truly operating a CIM system. 
The evaluator had seen many CIM systems, but this is the first one that 
really combines all major components of CIM, such as managerial, 
technical and manufacturing, and engineering; 2) Institutional and 
local support for the program were great; and 3) Interest and abilities 
of participating staff were great. 

One recommendation that the evaluator made during the first 
evaluation site visit was that communication among project staff 
members be improved. There needed to be more of an understanding of 
all of the components of CIM. 

Communication had improved by the time of the evaluator's second 
visit. For instance, business instructors were using shop floor 
language and vice versa. 

Dissemination has taken place on a variety of levels. Videos were 
produced with grant funds that explain CIM in several different ways. 
One video is of the overall CIM program, and all subsequent videos 
explain CIM*s application in business, design, publishing, and 
manufacturing. Presentations about the CIM program were also made at a 
Leadership 2000 seminar and a Society of Manufacturing Engineers 
conference in Chicago. Also, a CIM Breakfast, at which one of the CIM 
videos was shown, was held for 100 local business people, politicians, 
and the press. 



West Virginia Department of Education 
Charleston, West Virginia 

The West Virginia Department of Education, Bureau of Vocational, 
Technical, and Adult Education is located in Charleston, West Virginia. 
This grant was unique in that the grantee oversaw two training programs 
offered by separate vocational centers within the state: the Marion 
County Vocational-Technical Center and Carver Career and Technical 
education Center. These programs, which were implemented as a result 
of the Cooperative Demonstration grant, involved training in two 
different technology areas: health and computers. The grant programs 
were located in Charleston and Fairmont, West Virginia, about two hours 
apart by car. Roy Thomas was the overall project director and 
coordinator, and there were local project directors at each site. 

Specific objectives of the project, as stated in the proposal, 
were to: 

■ Implement cooperative training programs in 
the occupational areas of ADA computer 
language use, respiratory therapy, and 
hospital pharmacy technician; 

■ Train or retrain the unemployed, the 
underemployed or other adults who need 
upgraded skills in these technical areas; 

■ Place 80 percent of the trainees in jobs for 
which they have been trained at the end of 
the project; 

■ Identify, develop, or adapt instructional 
materials for program use; 

I Produce process and evaluative reports at the 
end of the project; and 

■ Disseminate project activities and results on 
a State and national level. 
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Marion County Technical Center (MCTC), located in Fairmont, West 
Virginia, developed and administered the ADA computer language program* 
This program became a part of MCTC*s existing robotics/automated 
technology program. An extensive curriculum, "A Secondary/ 
Postsecondary Curriculum for the ADA Programming Language" was 
developed as part of the grant. The curriculum as originally proposed 
would serve only adult technical students and industry personnel; 
however, as developed, it serves secondary students as well. 

Another change in the original plan for this program was the 
insufficient knowledge-base of members of local industry. They had to 
be taught the value of ADA if the training was to have any value to 
them. Their lack of familiarity with DOS was a hindrance to using ADA. 
Program staff, therefore, taught "DOS for Managers" ^nd developed a 
manual to accompany the training workshop. 

Marion County teamed with J&S Machine Corporation, which donated 
equipment and acted as a liaison to local industry. Ties with West 
Virginia University and Fairmont University, which assisted in ADA 
language curriculum development, also were established. Four business 
and industry workshops were taught during the project, two ADA 
workshops were held for teachers across the State, and one ADA 
teleconference for teachers also was held at Fairmont State University. 

ADA courses offered at MCTC were held for 18 six-hour sessions 
during a six-month period. The ADA program, which employed one new 
teacher who also was the main curriculum developer, proposed to serve 
10 students at MCTC and 10 business and industry personnel. This goal 
was met, although training provided to 34 business and industry workers 
was not in ADA as originally proposed, but in other computer 
technologies and computer-assisted drafting and manufacturing. ADA 
in-service training, however, was provided to 30 teachers. Eight 
students completed the MCTC training. 

Carver Career Center developed pharmacy technician and respiratory 
therapy technician training programs. The idea for these programs was 



already in motion, although actual partnership had not begun prior to 
the grant* Carver teamed with six health care providers, including the 
Charleston Area Medical Center (CAMC), which donated equipment and 
staff to serve on the advisory board* Existing curricula—from the 
California College Tor Health Sciences and the Michigan Pharmacists 
Association—were adopted with minor adaptations* A library and 
laboratory were set up at Carver for the respiratory therapy program, 
and Carver has an application pending for national certification of 
this program as well as State certification for the pharmacy technician 
program* 

An eleven month instructional program was conducted for the 
respiratory therapy program, which consisted of both classroom and 
clinical experiences* Students in the pharmacy program chose one of 
two five-and-a-half month training programs, which also involved 
classroom as well as clinical components* 

The goal for the respiratory program, which employed two new 
teachers, was to train 30 students* Thirty-four students enrolled, and 
24 of them completed the program* The goal for the pharmacy program, 
which employed one new teacher, also was to train 30 students* Twenty- 
nine students were enrolled, and 23 completed the program* 

Dissemination activities of the grant included one television 
report and five newspaper articles* ADA workshops and curriculum 
guides, as mentioned above, contributed significantly to successful 
dissemination practices* The Carver Career and Technical Center staff 
developed and distributed recruitment and promotion brochures of the 
health care programs* Roy Thomas made presentations at conferences of 
the National Association of State Directors of Vocational Education, 
the American Vocational Association Conference, and at the annual 
conference of the National Association for Program Improvement in 
Vocational Education* Copies of the final report and all curriculum 
materials are being presented to all vocational directors in West 
Virginia as well as to thG National Center for Research in Vocational 
Education, the ERIC system, the East Central Curriculum Center, and the 



West Virginia Curriculum Center. Copies of the final report are being 
distributed to all State directors of vocational education. 
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SUMMARY OF SURVEY OF FY1989 PROJECTS 



This appendix presents detailed findings from a survey of the 
FY1989 grantees under the Cooperative Demonstration Program (High 
Technology) of the 1984 Carl D. Perkins Act. All 30 FY1989 grantees 
were sent the survey, and, as of November 1991, responses (including 
telephone data retrieval) were obtained from 26 grantees. The grantees 
responding to the survey are identifed with an asterisk on the list of 
FY1989 grantees presented in Appendix B-2. In general, respondents 
were the individuals responsible for day-to-day management of the 
project. Although 26 grantees returned questionnaires, results 
reported here usually total 27 because one grantee was a state 
education agency that, in turn, dispensed the funds to two, unrelated, 
projects. 

The survey was designed to collect information on project 
implementation, especially factors related to startup, goals, major 
activities, partner involvement, and levels and types of services. 
Survey questions pertaining to project costs were discussed in the 
cost-benefit analysis in Section IV. Many of the survey questions were 
generated from findings of case studies in eight FY1988 grantees 
visited during the 1990-91 school year. In addition to describing 
programs, the questionnaire was designed to provide a framework for 
subsequent site visits. 

Instructions for the survey were distributed in stages to 
respondents. In early February 1991, all project directors received a 
detailed memorandum from the Office of Vocational and Adult Education 
(OVAE) indicating the information about clients, stc^ffing and extent of 
training that would be requested in the questionnaire. The memo was 
followed by phone calls from the study team to all grantees over the 
next month to reiterate thp need to obtain the data and to answer any 
questions grantees might have about the memo. The survey was mailed to 
grantees in July 1991, and telephone calls were made to obtain 
responses continued throughout the summer and fall. 

Despite the early meoio and phone calls, and repeated telephone 
follow-ups to the survey mailing, only 26 of the 30 grantees returned 
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questionnaires, and about one-third of the respondents provided 
incomplete systematic information,^ As a result, it is not possible 
to provide findings for the course completion and student outcome data 
requested by the survey. To facilitate the narrative discussion, 
simple frequencies are not presented as tables in the text. 

Training Areas 

The legislation mandating the Cooperative Demonstration Program 
allowed for projects in a wide range of industries, but most of the 
projects that were funded under the demonstration offered training in a 
few industries. Manufacturing was the most common industry focus, 
although health care and business services also were training areas 
(see questionnaire, item 7). Seventeen of the respondents reported 
manufacturing to be an area in which students were trained, seven 
projects indicated health care to be an area of training, and five 
reported a business services focus. Seven respondents indicated that 
training took place in multiple industries, and to specify a few 
industries would be misleading.^ 

Projects were divided almost equally between those that saw 
training of students to be their "key" activity and those that chose 
various activities associated with the development of the training to 
be most important (see questionnaire, item 6). Thirteen grantees 
identified student training itself to be the most important project 
activity, while five selected job skills identification, and five 
selected curriculum development as the dominant activity. Student 
recruitment, staff development, industry/occupational change, and 
institutional capacity building each were selected by one respondent. 

All projects provided occupation-specific training, and the 
majority (16 of 27) provided non-occupation-specific training as well 
(see questionnaire, items 26 and 31). The most common forms of non- 
occupation-specific training were provision of employabil ity skills (13 
projects), provision of basic academic skills, (11 projects), and 
provision of advanced academic skills (nine projects). 

Offerings were designed to teach both entry level and more 
advanced skills (see questionnaire, item 8) . Although eight grantees 
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reported that training was geared entirely to entry-level work and five 
reported that it was geared entirely to upgrading skills, 14 grantees 
reported that training was designed for both. Grantees providing 
training in health care were slightly more likely to emphasize entry- 
level skills and slightly less likely to emphasize upgrading or both 
types than other grantees. None of the programs training in business 
subjects taught entry-level skills entirely (see Table F-1). 

Overall, projects were somewhat more likely to be stand-alone 
efforts than parts of existing training programs at grantee 
Institutions (see questionnaire, item 9). Eight grantees indicated 
that their projects were part of ongoing training efforts, while 14 
indicated that the grants supported separate training programs. An 
additional five grantees described the^r efforts as both stand alone 
and parts of ongoing training programs.^ Projects training in 
construction, manufacturing, or multiple industries were somewhat more 
likely than those training in other fields to be separate from regular 
institutional offerings (see Table F-2). Ten of the 17 programs that 
indicated manufacturing as an area of training also indicated that the 
training was separate from other offerings at the institution. 

Although they may not have been tied to ongoing offerings, 
projects were likely to serve groups that previously had been served by 
the institution (see questionnaire, item lU). Only five grantees 
reported that the project served a new or different group exclusively, 
and eight indicated that the project was intended to benefit groups 
similar to those served previously. Fourteen projects indicated that 
both new and similar groups benefitted. Projects training in business 
services were the most likely to indicate that new groups benefitted 
exclusively (see Table F-3). Projects that provided training in health 
care were the most likely to indicate that groups similar to those 
served previously were served through this project. 

Projects characterized the likely "rewards" of training for 
participants as occupational in nature (see questionnaire, item 65). 
According to the grantees, the most common student reward was likely to 
be a job in a particular field (22 projects). Beyond that choice. 
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Table F-1 

MAJOR INDUSTRIES OF TRAINING AND SKILL LEVEL OF TRAINING PROVIDED 

(NUMBER OF PROJECTS) 



Number of 
Industry* Projects* 


Entry-level Upgrade 

Skills Skills Both 


Construction 4 
Manufacturing 17 
Business Services 5 
Health Care 7 
Education 4 
Multiple Industries 7 


0 1 3 
3 3 11 

0 1 4 
3 1 3 

1 1 2 
0 2 5 



* Industries with fewer than four responses omitted from analysis. 
Adds to more than 27 because projects could select more than one 
industry. 



Table F-2 



MAIN INDUSTRIES AND PLACEMENT OF PROJECT 
WITHIN GRANTEE INSTITUTION 
(NUMBER OF PROJECTS) 



Number of 
Industry* Projects* 


Addition to Separate 
Ongoing Training 
Training Program Both 


Construction 4 
Manufacturing 17 
Transportation 4 
Business Services 5 
Health Care 7 
Education 4 
Multiple Industries 7 


0 4 0 
3 10 4 

1 2 1 

2 1 2 
2 4 1 
2 2 0 
1 5 1 



] Industries with fewer than four responses omitted. 

Adds to more than 27 because projects could select more than one 
industry. 



Table F-3 



MAIN INDUSTRIES AND SERVICE TO 
NEW OR SIMILAR INSTITUTIONAL CLIENTELES 
(NUMBER OF PROJECTS) 



Number of 
Industry* Projects' 


Project Benefits 

Similar 
New or Group to 
Different Previously 
Group Served Both 


Construction 4 
Manufacturing 17 
Transportation 4 
Business Services 5 
Health Care 7 
Education 4 
Multiple Industries 7 


1 2 1 
4 2 11 

1 0 3 

2 1 2 
2 4 1 

1 1 2 

2 1 4 



* Industries with fewer than four responses omitted. 

Adds to more than 27 because projects could select more than one 
industry. 
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however, the most commonly given response was not on the questionnaire. 
Nineteen respondents wrote in responses that identified some form of 
behavioral training objective—most saying that students would be able 
to perform a particular set of occupational skills. Promotions in a 
particular field or with a particular employer were anticipated by 17 
and 15 projects respectively. Formal education credentials were 
somewhat less popular possible student rewards, with 14 projects 
anticipating vocational certificates, and 11 each selecting a 
degree/diploma or acceptance into further education. 

From an open-ended question on project beneficiaries, the study 
team organized projects by several primary types or classes of 
participants. These types were not mutually exclusive but were 
intended as general characterizations of project clienteles. They 
included: current employees of companies (13 projects), adults seeking 
work in a field or company (seven projects), disadvantaged adults (four 
projects) and high school students (three projects). For six projects, 
a second-level client was also characterized. Two of these projects 
indicated that second-level beneficiaries included regular college 
students, two indicated high school students, and one each indicated 
schoolteachers or disadvantaged persons. Two projects also identified 
a third type of client. 

Projects providing training in manufacturing were the most likely 
to indicate current employees as their primary clientele (see Table F- 
4). Fifty-nine percent (or 10 of 17) of such projects indicated that 
current employees were their primary clients. Programs providing 
training in multiple industries also indicated that th^y served 
existing employees primarily (four of seven, or 57 percent). Projects 
providing health care training were more likely to indicate that they 
served adults seeking work in a field or company (three of seven), 
which fits with the earlier finding that such projects were somewhat 
more likely than others to provide entry-level training exclusively. 
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Only one project providing health care indicated that it served current 
employees of companies. Projects offering training in business 
services were somewhat more likely than others to indicated that they 
served disadvantaged adults (two of five projects)/ 

Projects training for manufacturing and multiple industries were 
also more likely to train persons employed full time (see Table F-5). 
Although information on employment status of clientele was not 
available for all such projects, of the 16 projects training students 
in manufacturing for which data exist, seven indicated that 76 percent 
to 100 percent of students were employed full time, and an additional 
three indicated full-time employment by 51 percent to 75 percent of 
students. In contrast, none of the six projects providing training in 
health care for which information is available indicated that 76 
percent to 100 percent of students were working full-time, and only one 
indicated that 51 percent to 75 percent were working full-time. 

From these findings, then, a rough pattern of projects begins to 
emerge. The most common offerings appear to be in the area of 
manufacturing. Training provided under the grant is relatively 
separate from regular offerings at the same institutions; the clientele 
typically is current employees (most likely full- or part-time 
employees in the fields to which the project is geared); and this 
population (current employees) is one that the institution has served 
previously. Training appears to be geared to achieving mastery of a 
specific set of occupational skills more than to obtaining formal 
credential s. 

Economic Conditions end Effects on Projects 

From reviewing the project files and the site visits to FY1988 
grantees, the study team anticipated that training aimed at jobs or 
promotions would be a major part of grantee activities. Project 
proposals often included job placement goals for participants, so the 
ability to find jobs for trainees would be of paramount importance. As 
a result, the questionnaire asked grantees about the economic climate 
in the communities in which they were located as well as in the 
specific fields for which training was provided. 
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Grantees indicated that economic conditions in their communities 
had deteriorated since they applied for grants (mid-1990), but few 
indicated that the changes had affected implementation of the grant 
(see questionnaire, items 1 and 2). About half the grantees (14) 
indicated that economic conditions in the community had deteriorated, 
and 11 grantees indicated that conditions remained about the same 
throughout the period* Yet when asked whether the decline had affected 
grant implementation (e.g., did it mean fewer jobs for students or 
problems for the institution?), only 10 of the 14 indicated an effect. 
Four said that the changes had resulted in fewer jobs for students, and 
four indicated that the changes resulted in problems for the 
institution* 

Grantees were slightly less likely to indicate that a decline in 
economic conditions in the specific industry or occupation in which 
training was provided had affected project operations (see 
questionnaire, items 3 and 4). Well over half the respondents (16) 
indicated deteriorating economic conditions in the specific fields of 
training, but only half of those indicating a negative economic change 
also reported that it affected project operations. Six of these cited 
poorer job prospects for students as an outcome, while three indicated 
a negative effect on the institution. Two grantees reported that 
negative changes in the industry created a positive opportunity for the 
project by increasing training needs, and one grantee indicated that 
better economic conditions in the industry led to fewer training 
opportunities at the institution. These last responses may reflect a 
common finding that bad economic times, although creating poor job 
prospects, can sometimes translate into increased enrollment in 
training programs. Not only do the unemployed seek training, but 
persons who are concerned about losing their jobs may also enroll. 

Deteriorating economic conditions appear to have been most common 
among the projects that provided training in manufacturing (see Table 
F-6). Among the grantees, 71 percent of those that provided training 
in manufacturing reported that economic conditions in the industry of 
training had deteriorated since the grant application. Among those 
grantees providing training in health care, only 29 percent reported 
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Table F-6 

MAIN INDUSTRIES AND LIKELIHOOD OF ECONOMIC CHANGE OVER PAST TWO YEARS 

(NUMBER OF PROJECTS REPORTING) 



Number of 
Industry^ Projects* 


Industry Conditions 

Remained 

Improved Deteriorated the Same 


Construction 4 
Manufacturing 17 
Transportation 4 
Business Services 5 
Health Care 7 
Education 

Multiple Industries 7 


0 1 3 
2 12 3 

0 2 2 

1 2 2 
1 2 4 

0 2 2 

1 4 2 



* Industries with fewer than four responses omitted from analysis. 

Adds to more than 27 because projects could select more than one industry. 
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deterioration, while among those providing business services, 40 
percent indicated economic deterioration in the industry. Because the 
individuals enrolled in projects offering manufacturing training were 
the most likely to be full-time workers (see Table F-5), however, the 
economic deterioration may not have translated into immediate job 
effects. 

Project Goals and High-Tech Focus 

In the first year of the evaluation, site visits revealed that 
grantees had a wide variety of goals, with a substantial subset of 
grantees concerned as much with curriculum, staff, or institutional 
development as with the direct provision of training. For FY1989 
grantees, there was a major change in Federal direction: only 
applicants promising a "high-technology" training focus were considered 
for awards. It was thought that this constraint might narrow the range 
of project goals, so several questions were included in the survey in 
an attempt to understand the relative importance of training and other 
goals as well as the nature of the "high-tech" focus itself. 

Once again, the grantees expressed a wide variety of goals. For 
the FY1989 grantees, provision of training was the single most 
important goal, but almost half the respondents selected another goal 
as most important (see questionnaire, item 5 and Table F-7). Of the 27 
respondents, 15 indicated that creating new or improved training was 
the first goal. An additional seven grantees reported that increasing 
access to training for special or otherwise underserved populations was 
most important (see Table F-8). Given the overall Federal objective to 
develop models of public/private cooperation, it was surprising that 
the establishment of partnerships was ranked first by only three 
respondents. Improving economic productivity was the primary goal of 
two grantees.^ 

The distinction between the projects with a training focus and 
those with other emphases appeared to persist as projects selected 
second ranked goals. Of the 12 grantees that did not select training 
as the first goal, only three selected it as second. Grantees that did 
select training as their primary goal selected the goal of increased 
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Table F-7 
RANKING OF PROJECT GOALS 





Project Goals 




increase 


tstaDi 1 sn 


Improve 




rroject Name 


Access 


Partner 


Training 


utner 


Alabama Aviation College 


2 




1 




Ben Hill -Irwin Institute 




2 


1 




Bronx Community College 


1 


2 






CORD 


2 




1 




Clackamas Community College 


2 


1 






Columbia Basin College 




2 


1 




Fox Va ; ley Tech 




2 


1 




Hampden County Consortium 


1 




2 




Home Builders Institute 


2 




1 




Howard Community College 


2 




1 




Illinois Eastern College 






1 




Indian Hills College 


i 


2 






John M. Patterson College 




2 


1 




LTV Steel Co. 




2 




1 


Luzerne Community College 


1 




2 




Nebraska Labor Department 


2 


1 






North Clackamas Schools 




2 


1 




Northampton Community College 




2 


1 




PAVE 




1 




2 


State Center College District 


2 




1 




Valencia Community College (Health) 




2 


1 




Valencia Community College (Tele) 


1 


2 






Valencia Community College (CIM) 


1 


2 






Waubonsee Community College 


1 




2 




West Vj.ginia Northern College 




2 


1 




West Virginia Department of 




2 


1 




Education 1 










West Virginia Department of 




2 


1 




Education 2 
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access to vocational training for special populations somewhat less as 
a second goal than did other grantees* Of the 15 that selected 
training first, only five selected increased access for special 
populations as their second goal. This group was more likely to choose 
public/private partnerships second, with 10 making that selection. 
What these choices suggest is that increased access of special 
populations was the primary or secondary goal of a distinct subset of 
projects--those that saw the goal of developing new or improved 
training as somewhat less important/ 

Projects that selected new or improved training as their main 
goal also differed from those that selected increased access in what 
they considered to be their "key" project activities. Projects that 
held new or improved training as their most important goal were more 
likely to view curriculum development as their key project activity 
(see Table F-8). Of the training projects, five of 15 saw curriculum 
development as the most important activity. In contrast, among the 
projects emphasizing increased access of special populations, none saw 
curriculum development as its most important project activity. The 
"increased access" programs were more likely to emphasize the training 
itself as the most important acti vity--with four of seven making this 
selection. 

As for the meaning of high technology, the projects also 
reflected different interpretations of the Congressional mandate. 
Based on case studies of FY1988 grantees, the study team identified 
four basic ways in which projects incorporated high technology in 
developing their programs and delivering training. Projects were high- 
tech if: 

■ Training was designed to prepare 
students for jobs in fields that 
manufacture high-tech products or 
service high-tech equipment; 

■ Training was designed to enable 
students to use high-tech equipment 
or products even though the field 
in which the equipment is used is 
not generally considered high-tech; 
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Training was conducted on high-tech 
equipment, such as computers, CAD, 
or CIM equipment; or 



I 



Training was offered in basic 
skills as preparation for specific 
occupational training in a high- 
tech field. 



In the survey, FY1989 grantees were asked to select all the high-tech 
definitions appropriate to their projects. 

Most grantees selected more than one response to describe their 
projects' high-tech elements (see questionnaire, item 13). Fourteen 
projects selected [a], 12 selected [b], 17 selected [c], and 13 
selected [d]. To make it possible to analyze these data further, the 
team observed the overlap between [a] and [b], which asked respondents 
to select essentially mutually exclusive fields of employment. The 
study team discovered that there were only three projects that selected 
both responses, and that a total of 23 projects had selected [a], [b], 
or [a] and [b]. Of the remaining four projects, one selected [c], two 
selected [d], and one selected [c] and [d]. The study team reasoned 
that projects selecting [a] or [b] and also making selections from 
among [c] or [d] were, in making their [c] and [d] selections, picking 
activities that occur as a function of training provided in fields 
described in [a] or [b]. Hence, for these projects the team would 
concentrate further analysis on the field selection— i .e. , [a] or [b]. 
As a result, the team created one response category for each project 
for fu'.ther analysis as follows: 



I 



[a], but not [b]: training for high 
tech field--ll respondents; 



I 



[b], but not [a]: training for 
high-tech equipment in non-high 
tech field— nine respondents; 



I 



[a] and [b]: training for both high 
tech field and for using high-tech 
equipment in a non-high-tech field- 
-three respondents; 



[c] only: training on high-tech 
equipment--one respondent; 
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■ [d] only: basic skills training to 
prepare for further training in a 
high-tech field--two respondents; 
and 

■ [c] and [d] only: training on high- 
tech equipment and basic skills 
training in preparation for further 
training in a high- tech field— one 
respondent/ 

Using these categories, most of the projects (23) defined "high- 
tech" in terms of the occupation or equipment for which training was 
provided. Of those projects, about half (the 11 [a]s) were explicitly 
preparing persons for immediate or specific work in high tech fields. 
Most of the others (the nine [b]s) were preparing students to use high- 
tech equipment or products in a non-high-tech field, with the rest 
doing both. If these answers are indicative, the remaining four 
projects were not preparing students for explicit high-tech 
applications, but rather defined their projects' high-tech element in 
terms of the equipment used in training or the students' long-range 
occupational goal. 

Second, the study team examined more closely the extent to which 
the 23 projects that defined their high-tech focus in terms of 
occupation or field application also used high-tech equipment or 
provided basic skills instruction as part of their activity (see Table 
F-9). The team found that of the programs that prepared students for 
work in high-tech fields, the vast majority (eight of 11 under [a] 
above) used high-tech equipment in training. Among the projects 
preparing students to use high-tech equipment in non-high-tech fields, 
however, only a slim majority (five of nine under [b]) conducted 
training on high-tech equipment. Because these projects were 
explicitly preparing students to use high-tech equipment or products on 
the job, the lack of training on such equipment in a majority of them 
is surprising. It suggests that the training may have been quite 
limited in scope. 

The projects preparing students to use high-tech equipment in 
non-high-tech settings were also more likely not to offer basic skills 
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Table F-9 

EXTENT TO WHICH PROJECTS THAT PREPARED STUDENTS TO WORK 
IN HIGH TECH FIELDS OR WITH HIGH TECH EQUIPMENT TRAINED 
STUDENTS ON HIGH TECH EQUIPMENT OR IN BASIC SKILLS 







Trained on High 
Tech Equipment 


Target Basic 


Skills 


i 


High Tech Definition 
of Project 


Yes No 


Yes 


No 


a. 


Prepared Students for Jobs 
in Fields Manufacturing or 
Servicing High Tech 
Products. (n=ll) 


8 3 


6 


5 


b. 


Prepared Students to Use 
High Tech Products 
Although Field Was Not 
High Tech. (n=9) 


5 4 


2 


7 


a. 


and b. (n=3) 


2 1 


2 


1 
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instruction. Only two of nine such projects indicated basic skills 
instruction, compared with six of the 11 projects preparing students 
for jobs in high tech fields. This finding also suggests that the 
training for use of high-tech equipment in non-high-tech fields may 
have been rather limited or narrow in scope. 

Projects that included manufacturing were considerably more 
likely than others to be preparing students for work in high-tech 
fields (see Table F-10). Eleven of 17 projects that indicated 
preparation for the manufacturing industry— 65 percent— selected either 
[a] or [a] and [b] combined. Projects described as training for 
multiple industries also were weighted to preparation for high-tech 
fields, with four of seven selecting this choice. This compares with 
only one of seven projects that provided preparation for health care 
jobs, with five of the seven respondents selecting [b]. None of the 
four projects that included training for the transportation industry 
provided training for a high-tech field, and business service projects 
were about equally divided between high-tech and non-high-tech fields. 

Project Clientele: Overview 

One of the important goals of the Perkins Act has been to 
increase the access of special populations to vocational training. 
While the specific mandate of the Cooperative Demonstration Program 
with respect to special populations is ambiguous, and the grants 
announcement is silent on the issue, the importance of the goal to the 
overall Act makes it worthy of examination. The questionnaire included 
a number of questions designed to describe the special populations that 
were served through Cooperative Demonstration projects. 

As noted previously, a subset of Cooperative Demonstration 
projects considered increased access of special populations as their 
most important goal. Further examination shows that these projects 
were also more likely to report that their project served a clientele 
that was new or different Tor the institution (see Table F-11). When 
asked whether the project benefitted a new or similar group of clients, 
three of the seven projects that selected the increased access goal as 
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Table F-11 

PROJECTS' PRIMARY GOAL AND LIKELIHOOD OF SERVING 
NEW OR DIFFERENT CLIENTELE'^ 



Primary Goal of Project 


Clientele Benefittina From Pro.iect 

New or Different Similar to Past 
Group for Groups Served 
Institution by Institution Both 


Increased Access for 
Special Popul ations 
(n-7) 

Create New or Improved 
Training (n=I5) 

Establish Partnerships 
(n=3) 

Economic Development 
(n-2) 


3 0 4 

1 7 7 

0 1 2 

1 0 1 
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primary also indicated that the project clientele was new or different. 
The other four projects with the access goal indicated that they served 
new groups as well as groups similar to those served in the past. None 
of these programs indicated that its project served a clientele 
entirely similar to groups served previously. By contrast, only one of 
the 15 projects that selected the improvement of training as its first 
goal also indicated that it served a new group exclusively, and seven 
indicated that they served groups that were entirely similar to those 
served in the past. An additional seven programs indicated that they 
served both new groups and groups similar to those served in the past. 

The projects that held increased access as their primary goal 
were also more likely to report that they serve disadvantaged persons 
(see Table F-12). Comparisons of project goals with responses to the 
open-ended question on project clientele indicated that of the seven 
projects with increased access as the main goal, three characterized 
their primary clientele as adults seeking work in a particular field or 
company, three characterized their clientele as disadvantaged persons, 
and one indicated that it served existing employees of companies. None 
of the programs that saw new or improved training as their primary goal 
indicated that they served disadvantaged persons primarily, with seven 
of 15 indicating current employees as their primary clients and the 
majority of the rest indicating adults seeking work. Or to put it 
another way, three of the four program serving disadvantaged persons as 
their primary clients also indicated that increased access was their 
main goal. Seven of the 13 projects serving current employees of 
companies primarily indicated that new or improved training was their 
main goal . 

Despite the rather limited number of projects that indicated they 
served special populations primarily, the vast majority of projects 
reported making specific efforts to recruit special populations (see 
questionnaire, item 24). All 19 projects that conducted any 
recruitment activities also reported such efforts, with the largest 
number trying to recruit women and blacks (18 projects). Hispanics and 
unemployed persons were a focus of recruitment efforts in 16 projects, 
followed by low-income persons (12 projects), and persons with 



F-24 



UJ 



o 



CSJ 

I 

25 CO 
CL. 3 



00 
UJ 

ex. 



O 00 
O -M 

x: c 

00 

JZ 
CD 00 









o 








CD 




Cl> 


cr> 




4-> 


<t3 


00 


£= 


-M 


c 


CD 


C 


o 


•r— 




00 


r~- 


> 




O 




CD 






> 


00 










fO 


O 




E 














CD 00 




Q. 














^ CP 


c 


t/) 


a> 


fO 




CD u. 


Q. 




CO 


E 


(1) 


c 


o 


•r-: 


00 




O 
















o 




-o o 






cac 3: 





00 

-M CD 
C CD 
CD 

O 

<^ E 



n3 +J 
O O 
C5 CD 

E 

o 



ro 



II 



00 
CD 
O 
O 

<c 

o 

00 
fT3 
CD 

u 
c 



CD 



fT3 



"O 
CD 
> 

o 

Q. 



O LO 
CD C 



CO 

CD 
C 



fTJ CO 



CD 

E 

Q. 
O 

CD 
> 
CD 



E 

o ^ 

C CM 

o tt 



c 
o 



00 

n 
cr 

"O 
CD 



c 

CD 
Q. 
O 

E 

o 



CD 
> 

• r— 

CD 

t:> 

00 
CD 

s- 
o 

CD 
CD 



F-25 



disabilities (11 projects). Other populations targeted by projects 
with less frequency included Asian or Pacific Islanders American 
Indians or Alaskan Natives, persons reading below the eighth grade 
level, immigrants, and persons living in rural areas. The methods used 
to recruit special populations varied, but distribution of posters, 
flyers, or other materials, advertisement in print or broadcast media, 
internal recruitment within the grantee organization/district, and 
contact with government offices (including PICs and employment 
services), were among the most popular. 

Although most projects made some efforts, the four projects that 
indicated they served disadvantaged students were more likely to have 
made extra efforts to recruit these groups (see Table F~I3). 
Interestingly, only four of the seven projects that served adults 
seeking work made extra efforts to attract special populations, 
suggesting that they may already have had a readily available source of 
such students. The three projects that served high school students 
also made efforts to attract special populations.® 

As noted previously, business services projects were somewhat 
more likely to indicated that they served a new group than were 
projecls training in manufacturing or health. It is hard to reach any 
conclusions about the relationship between primary clientele and 
industry of training, however, because only four projects identified 
disadvantaged persons as their primary clientele. If the definition of 
"aisadvantaged" is broadened to include adults seeking work in a field 
or company, however, a pattern of relationships between industry and 
clientele served begins to emerge. 

If projects serving disadvantaged persons and adults seeking work 
are combined, the survey shows that projects in areas other than 
manufacturing or "multiple industries" were more likely to serve these 
populations (see Table 4). Three of five projects providing training 
in business services, three of four projects with training for 
transportation, and five of seven offering health care training 
primarily served these groups (combined). On the other hand, only six 
of 17 projects providing training for manufacturing and three of seven 
that trained in multiple industries served these groups primarily. 



F-26 



Table F-13 

PROJECTS' PRIMARY CLIENTELE AND LIKELIHOOD OF SPECIAL- 
POPULATION RECRUITMENT EFFORTS 
(NUMBER OF PROJECTS) 



J_ Primary Clientele* 


Special Population Recruitment Effort 
Yes No 


Current Employees of Companies 

AduHs See/ving Work in Field or 
Comp-pny 

0^ ..advantaged Persons 

|i 

|[High School Students 


8 5 
4 3 

4 0 

3 0 



Categories derived from open ended question. 
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The good news, then, is that the projects most likely to serve 
special populations were also those conducting training in fields less 
affected by the economic downturn. Whether this occurred by design or 
by default is unknown, however. Was it the initial plan of the 
manufacturing training projects to focus on current employees, or did 
projects offering this training adapt to a poor job market by serving 
those already employed? Did projects offering training in health care 
or business decide to focus on persons seeking work or disadvantaged 
persons because they believed jobs were available? 

Projects serving adults seeking work and di.sadvantaged persons 
also appeared somewhat more likely to provide training aimed at the 
development of skills for entry-level work (see Table F-14). Three of 
the 11 projects (27 percent) aimed at the two groups combined reported 
providing entry-level skills traimng exclusively, with the eight 
remaining projects indicating that they provided training for both 
entry-level and upgraded skills. None of these projects reported 
providing instruction aimed exclusively at upgrading skills. Of the 13 
projects serving current employees, three (23 percent) indicated that 
they provided entry-level skills exclusively, but four (31 percent) 
indicated that they provided upgrade skills exclusively. The six other 
projects indicated that they provided both. 

Given the finding that projects serving adults seeking employment 
and disadvantaged persons tended to offer training in fields other than 
manufacturing, and the earlier finding that manufacturing projects were 
more likely to prepare students for work in high tech fields, it is not 
surprising to find that disadvantaged persons and adults seeking 
employment also were less likely to train for high-tech fields (see 
Table F-15). Of the 11 projects that emphasized these populations, 
seven were preparing students to use high-tech equipment or products in 
fields not generally considered high-tech. Only three were preparing 
students to work in high-tech fields, and one indicated that it was 
using high-tech equipment in training and providing training in basic 
skills. In contrast nine of 13 projects for current employees were 
training students for work in high tech fields or a combination of 
high-tech and non-high-tech fields. 
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Table F-14 

PRIMARY CLIENTELE AND SKILL LEVELS OF TRAINING 
(NUMBER OF PROJECTS) 



rn nidry 
CI ientele 


Entry-1 evel 
Skills 


Upgrade 
Skills 


Both 


Current Employees of 
Companies 


3 


4 


6 


Adults Seeking Work in 
Field 


1 


0 


6 


Disadvantaged Persons 


2 


0 


2 


High School Students 


2 


1 


0 



Categories derived from open ended questions. 
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Projects aimed at disadvantaged persons and adults seeking work 
also were more likely to provide non-occupation specific training in 
addition to occupation specific training (see Table F-15). All four 
projects for disadvantaged persons provided non-occupation specific 
training as did five of the seven projects for adults seeking work. In 
contrast, only five of the 13 projects for current employees provided 
such training. This finding may indicate that the current employees 
were considered by project staff to have sufficient background to 
pursue occupational training without additional assistance, or that the 
training provided was relatively narrow in scope and, hence, required 
little supplementary instructs i. 

Pro.iect Clientele: Information Drawn from Participant Counts 

In addition to characterizing their overall clientele, projects 
were asked specific information about participants. These included 
participants' age, race, sex, proficiency in English, special 
population status, level of education attained, primary educational or 
vocational goal, employment status at the time of training, training 
status at the end of the grant period, and employment outcomes. 
Unfortunately, fewer than half the projects were able to supply 
information beyond total number of participants, sex, race, and 
employment status during training. The projects that failed to supply 
participant information also were those that claimed the largest 
numbers of participants. Nonetheless, some findings can be reported. 

The total number of participants varied widely by site (see 
questionnaire, item 22). Twenty-six projects provided head count 
enrollments (not necessarily undupl icated) that ranged from nine to 
1,589, with a median enrollment of 113 students. Only 14 projects 
also were able to supply PTE enrollment figures, enrollments that 
ranged from 19 to 517, with a median of 30 students. 

Males were the largest number of training recipients (see Table 
F-17, and questionnaire table 1) . For the 79 percent of participants 
for whom sex data were provided, 57 percent were male, and 33 percent 
were female. Further, males were concentrated in the programs 
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Table F-16 

PROJECTS' PRIMARY CLIENTELE AND LIKELIHOOD OF PROVIDING 
NON OCCUPATION-SPECIFIC INSTRUCTION 



Primary Clientele* 


Non-Occupationall V Specific Instruction 
Yes No 


Current Employees of Companies 

Adults Seeking Work in Field 
or Company 

Disadvantaged Persons 
High School Students 


5 8 
5 2 

4 0 
2 1 



Categories derived from open ended question. 
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providing training in manufacturing, while females were concentrated in 
the programs providing training in business services and health care 
(see Table F-18). Although 19 programs indicated that they made 
special efforts to attract women, it would not appear that the projects 
provided a testing ground for nontraditional training by sex. 

For the 21 projects reporting the race of most or all trainees, 
the vast majority were white (see Table F-17). In these projects, 
whites constituted approximately 83 percent of participants for whom 
race was reported, blacks were 11 percent, Hispanics were five percent, 
Asians were one percent, and American Indians/Alaskan Natives were less 
than one percent. Fewer than two percent of participants were limited 
English proficient. Overall, whites constituted an even greater 
percentage of participants in these projects than repr^esented in the 
population. So, although 19 grantees said that they n;?id€ special 
efforts to recruit minorities, among the 21 projects that supplied 
information, the program appears to have done little to demonstrate new 
opportunities for minorities. The only caveat is that the seven 
projects with missing data on race include about half the 
participants 

The vast majority of trainees were employed full-time during 
training. Of the 87 percent of participants for whom employment status 
was reported, 85 percent were employed full-time during the period of 
training (see questionnaire table 1). Further, most of these persons 
were employed by the private-sector partners that cooperated in the 
projects. Of the 80 percent of full-time employees for whom the 
employer was known, 92 percent were employed by a partner company. The 
total percentage employed by partners was probably somewhat less than 
this figure, of course, because employment by the partner was more 
likely to be known to the grantee. Nonetheless, one can safely 
conclude that the main training recipients were full-time employees of 
partner organizations. 

Understanding the employment status of the clientele makes it 
easier to explain the responses to a question that asked grantee: to 
indicate the number of participants seeking various educational r 
vocational outcomes from the training (see questionnaire table 1 . 
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Choices provided to respondents included job placement, job promotion, 
vocational certificate, high school diploma or GEO, college credit, 
associate degree, and "other." The most popular educational outcomes 
for participants reported by grantees was a write-in category that 
indicated 36 percent of students sought specific job skills. The 
second most common educational goal was "unknown": 19 percent of 
students; followed by job promotion: 17 percent of students; and job 
placement: 14 percent of students. Projects reported that nine percent 
of project clients sought college or CEU credits. Either most students 
did not seek credentials or programs did not provide them and, hence, 
did not know whether students were seeking them. Only six percent of 
students were reported as seeking any credential (including a high 
school diploma or GEO, vocational certificate, or associate degree). 
Combined with the findings that most participants were employed, and 
that mastering a set of oc"'. national skills was one of the most popular 
likely "rewards" for students from the projects' perspective, these 
findings suggest that projects were providing forms of training 
tailored to very specific job requirements. 

Project Activities 

In order to understand how projects operated, the questionnaire 
asked respondents to identify the main project activities and the 
amounts of staff time devoted to each. Grantees were asked about 
activities conducted as part of project development as well as direct 
services to clients. The responses to these questions showed the 
tremendous diversity of emphases and approaches undertaken through the 
demonstrations. The variety of responses also points up the problems 
inherent in attempting to compare implementation across such disparate 
interventions. 

Preparatory Activities . There are great ranges in relative 
emphasis in the activities undertaken by projects in preparation for 
offering services to clients. The most common preparatory activity was 
to identify the skills or training needed in a particular field or 
geographic area (see questionnaire, item 18). Twenty~two projects 
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indicated that they carried out this activity, but the staff hours 
devoted to the activity ranged from 10 to 2,234. Across all projects, 
the mean number of hours was 272 and the median was 175. Other time- 
consuming start-up activities included: 

■ Recruiting and hiring staff (21 projects, 
median hours: 50); 

■ Development of student assessment materials 
(20 projects, median hours: 65); 

■ Recruiting students (19 projects, median 
hours: 100); 

■ Review of planned offerings by experts, 
potential employers or institutional 
officials (16 projects, median hours: 50); 

■ Recruiting employers interested in having 
employees trained or hiring students (16 
projects, median hours: 85); and 

■ Contracting for training (15 projects, median 
hours: 50). 

Some teacher training was undertaken in most of the projects, but 
the amount of training varied widely (see questionnaire, items 38 and 
39). Sixteen projects indicated that they provided instructors or 
other staff with pre-service or in-service training in order to teach 
in the project. An additional eight projects reported that such 
training was not necessary. Two projects did not provide training, but 
thought in hindsight, that it would have been valuable. Pre-service 
training for instructors occurred in 12 projects and varied from six to 
763 hours per project (number of staff trained times number of 
instructional hours), with a mean of 158 hours but a median of only 60 
hours. In-service training occurred in 11 projects and ranged from 10 
to 2,000 hours, with a mean of 357 and a median of 200 hours. What 
this suggests is that a few projects spent a great deal of time on pre- 
and in-service training, while the majority spent little. 

Curriculum Development . As noted previously, five projects 
considered curriculum development to be their most important activity, 
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but every project reported that it spent some time selecting a 
curriculum (see questionnaire, item 34). Ten projects reported that 
the curriculum was developed entirely by project staff, while nine said 
their curriculum was adapted or adopted from a curriculum already in 
use within the grantee institution. One project indicated that it 
adopted or adapted a curriculum in use at another institution, and one 
other adapted/adopted a curriculum already used in industry. Six 
projects used a combination of approaches. Given that only one project 
used a curriculum adopted/adapted from another institution, it would 
appear that replicating an approach tried elsewhere was not a major 
emphasis in this demonstration program. 

Overall, projects devoted considerable time and effort to 
curriculum development (see questionnaire, item 18). Staff hours 
ranged from eight to 3,000, with an average of 539 but a considerably 
lower median number of 241 hours. Twenty-two projects indicated that, 
as part of curriculum development, they worked with private industry to 
identify training needs and skill requirements. Most also worked with 
industry to design the curriculum (21 projects), gain approval (19 
projects), and pretest and revise the curriculum (16 projects). Five 
projects also consulted with union officials in developing curriculum. 

As part of program development, almost all of the projects (24) 
sought to identify the specific job skills or qualities sought by 
employers (see questionnaire, item 33). To identify those skills the 
projects interviewed practitioners in the field (21 projects), surveyed 
employers (20 projects), reviewed standard or existing skill- 
requirement information (19 projects), observed skills at work sites 
(16 projects), reviewed employer job descriptions (16 projects), and 
conferred with experts (15 projects). In addition, nine projects 
reviewed previous reports and/or studies in an effort to identify the 
job skills or qualities employers sought. 

The primary goal of the project did not appear to be a major 
determinant of whether the curriculum was developed anew or 
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adapted/adopted (see Table F-19). Projects having a primary goal of 
improved training weie somewhat more likely to develop new curricula 
than projects primarily concerned with increased access, but the 
differences were not large. 

Training . In general, grant-supported training was of short 
duration (see questionnaire table 1). Grantees reported that 68 
percent of participants received between one and 100 hours, while 11 
percent received 101 and 250 hours (see Table F-20). Approximately 14 
percent of participants received between 251 and 1,000 hours of 
training and 14 percent received 1,000 or more hours of training. 
There is a r jor caveat with respect to the hour totals, however~-one 
that suggests that the actual number of grant-supported hours of 
training may be lower than these numbers reflect. Not all projects 
interpreted total instructional hours received through the orant in the 
same manner. The questionnaire instructions asked for "hours of 
instruction. . .from the project during the grant period," but it is 
likely that at least some projects included total instructional hours 
participants received, whether or not the training was project 
supported. For example, a project might have provided revised or 
additional services to an ongoing course or degree program. Some sites 
reported the total number of hours required to complete that course or 
degree program as the instructional hours received under the project. 
Other projects— especially those where grants largely supported 
separate training— reported only the portion of the course or program 
hours affected by a grant-supported activity or curriculum. 

Attempts to obtain data on the methods of instructional delivery 
yielded few responses, and these reflected a wide variety of 
approaches. Open-ended questions that asked about the type of 
instruction students received (e.g., lecture, lab) and other services 
provided (e.g., counseling) were answered by only 13 and 11 projects, 
respectively, and the responses varied widely in terms of service modes 
(see questionnaire, item called table 1). Among the responding 
projects, applied learning was quite common, with some combination of 
laboratory and lecture formats popular. 
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In addition to occupation-specific training, 16 projects included 
non-occupation specific instruction or training as part of their 
program (see questionnaire, item 31). The non-occupation-specific 
skills taught most often included employabil ity skills (13 projects), 
basic or remedial academic skills (11 projects), or advanced academic 
skills (nine projects). ^° 

Student Assessment . All but two projects reported that they 
conducted some type of student assessment, but few conducted the same 
assessments at entrance and later in the training (see questionnaire, 
item 32). The most common method was assessment of job skills, with 20 
projects reporting this method. Of those 20 projects, 12 assessed job 
skills at entrance, and 17 assessed job skills once instruction had 
begun. Nine projects conducted job skill assessments both at entrance 
and during training. The second most common form of assessment was the 
administration of an academic aptitude test, which 15 projects gave at 
entrance. Only three of the 15 projects administered this form of test 
again after training had begun. 

In fact, beyond job skill, the assessments favored at entrance and 
during training were different. As noted, the use of academic aptitude 
tests dropped off after entrance. In contrast, the use of criterion- 
referenced achievement tests increased. Seven projects administered 
criterion-referenced tests at entrance, but 11 administered them once 
training had begun. Only five projects administered criterion- 
referenced tests both at entrance and during training, however. 
Vocational aptitude tests were used by nine projects at admission, but 
only three projects thereafter. Eight projects administered a 
standardized achievement test at entrance, but only one project 
administered such a test thereafter. The general lack of consistency 
between baseline and subsequent student assessments suggests that, with 
the exception of nine projects that conducted baseline and follow-up 
assessments of job skills, projects would be unable to determine their 
impact on students' performance. Given that these projects were 
intended to be demonstrations, the built-in inability to provide 
evidence of effectiveness is most disheartening. 
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Support Services > Based on the findings from the first year of 
the evaluation, the study team anticipated that projects would offer 
participants services in addition to training. To determine the range 
of services, projects were asked about the types of services they 
offered directly, as well as those provided by another agency in 
coordination with the project (see questionnaire, item 20). Overall, 
21 projects indicated that they provided support services directly or 
coordinated with another agency to provide the service. 

Aside from student assessment and academic remediation, which have 
been discussed already, the most common services were those that 
involved career information. Employabil ity advice was available in 20 
projects—including 13 that provided it directly and through 
coordination, as well as 10 that provided it entirely through 
coordination. Also commonly available were career or other counseling, 
which 17 projects provided. Of those, 15 provided it directly, and two 
provided it in coordination with another agency. Three projects 
offered both. Job placement services were available in 16 projects, 
with 13 providing the service directly. The only other service 
available in more than half of the projects was tutoring, which was 
available in 15 projects and directly provided by 12. The nature of 
the support services overall appears to reflect the fairly job-specific 
focus of the projects. 

Given the rather small amount of training per student and a 
clientele that was for the most part employed full-time, a substantial 
minority of projects offered students financial assistance. Eight 
projects indicated that they provided stipends or other financial aid 
directly to students. Only one of these projects reported that 
students were eligible for financial aid through another source. Six 
projects said they provided students with transportation assistance 
directly, and one coordinated assistance. Four projects provided child 
care directly, and five projects offered health care directly or 
referral for health care. 
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Public/Private Cooperation 

The Cooperative Demonstration Program was designed to encourage 
cooperation between educational institutions and the private sector. 
As the grants announcement noted: 

High technology training can be conducted most 
effectively with the active involvement and 
cooperation of the private sector. Effective 
partnerships between the private sector and public 
agencies in vocational education are an important 
aspect of the Cooperative Demonstration Program 

The announcement anticipated that the partnerships established by the 
project would provide models of effective cooperation. To understand 
the nature and extent of cooperation, the survey asked numerous 
questions about the projects' public/private partnerships. 

Most, but not all, projects established relationships with 
employers (see questionnaire, item 40). Of the 27 respondents, 23 
indicated some involvement of employers in planning or administering 
the project or in providing services. Of the remaining four projects, 
only one indicated that no outside partnership was formed. Some 
projects had multiple types of partners including schools or 
universities (institutions that might be employers or may be involved 
in the project in some other manner-- 19 projects) and community-based 
organizations (10 projects). 

Although employers were involved in 23 projects, not all of these 
projects indicated that private businesses were the most important 
partner organizations (see questionnaire, item 42). Slightly more than 
half the projects (14 of 26) indicated that the most important partner 
was a private business. Of the remaining projects, five said that the 
most important partner was a trade association or consortium, four 
indicated an educational institution, and three indicated a non- 
educational public agency. It would appear that, in at least the five 
projects linked with an association or consortium, the most important 
private-sector partner was not an institution capable of employing 
persons trained by the program. For grantees that had second partners 
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(see questionnaire, item 51), these partners were even less likely to 
be private businesses (nine of 20) and more likely to be trade 
associations (four of 20). 

Although most grantees formed partnerships with only a few 
organizations, some established multiple partnerships (see 
questionnaire, item 41). Six projects had one private sector partner, 
and 16 projects had four or fewer, but seven projects indicated that 
they had 10 or more partners. Given the tremendous range in the number 
of partners, it is likely that the roles of partners differed 
considerably across the projects. 

In fact, differences in partner contributions were considerable. 
When asked to rank in importance a list of possible activities for the 
most important partner organization, there was little consensus among 
the grantees (see questionnaire, item 44). The most commonly selected 
first choice activity for the partner organization was to provide 
equipment for training, but that choice was selected by only seven 
grantees. Five grantees said that the partner's main contribution was 
identifying job skills, while four indicated that the partner recruited 
students for training. Three indicated that the partner served on a 
project advisory committee, and two indicated that the partner supplied 
instructors. In short, there was considerable variability in first 
partner contribution— a variability that is sustained when the first- 
through third-ranked activities are combined. The same findings hold 
true for second partners, and, in fact, the variability in contribution 
increases (see questionnaire, item 52). 

The differences in activities may be, in part, a function of the 
nature of the partner relationships. Based on categories developed 
from site visits to FY1988 grantees, the study team identified several 
ways to characterize or summarize the relationships between grantees 
and employers. Each FY1989 grantee was asked to pick the choice that 
best characterized its relationship with its first partner (see 
questionnaire, item 45). The majority of grantees selected one of two 
choices: 
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■ The partner was a customer of the project, 
e.g., the project provided customized 
training to the partners' employees (selected 
by nine grantees) ; or 

■ The partner participated in the delivery of 
instruction and services (selected by nine 
grantees) 

No other choice was selected by more than three grantees (see Table F- 
21). The nature of the relationship was then matched with the most 
important partner activity. 

In most of the projects, the grant did not result in the 
establishment of a new primary partnership (see questionnaire, item 
46). Nineteen of 27 grantees reported that the relationship between 
the grantee institution and the partner organization was not new, but 
all 19 also said that the relationship had been strengthened as a 
result of the project. The eight projects that did establish new 
relationships with the first partner organization were more likely to 
have encountered problems. Three of six projects that reported any 
problems with their first partner had new partners, and two of these 
partnerships were dissolved at the end of the grant. 

Grantees were slightly more likely to have established new 
partnerships with their second partners (see questionnaire, item 54). 
Eight of 21 second partners were new, and appeared to reflect a wider 
range of relationships. Considerably fewer second partners (2) were 
project customers (i.e., they purchased customized training or hired 
students), though a sizeable number (eight) shared in delivery of 
instruction and services. This difference between first and second 
partners may have occurred because second partners were more likely to 
be trade associations or other educational institutions rather than 
employers. Second partners were somewhat more likely to serve 
primarily as members of advisory committees. 

Grant Operations 

With any Federal demonstration program, there is interest in 
whether funds provide an opportunity that would not otherwise have been 
available and do not simply substitute for existing resources. It also 
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is hoped that Federal funds will provide the impetus for funding from 
other sources so that activities can be expanded or continued after the 
Federal support has ended. To address these policy issues, respondents 
were asked about the importance of Federal funds in project startup and 
development, as well as the likelihood of project continuation beyond 
the grant period. 

In addition, based on the examination of the FY1989 grantees, 
there was a specific concern about the ability of grantees to start up 
quickly and to complete their projects. In general, the lengthy period 
between proposal submission and award notification, the mid-academic- 
year starting date for most grants, and the 18-month duration of the 
grant all contributed to startup and completion problems for some of 
the grantees. For example, when they submitted their proposals, 
several projects had identified persons to be hired, but those persons 
had taken other jobs by the time the awards were made. Finding new 
staff who were willing to work on a closed-end, 18-month grant took 
considerable time, and several FY1988 projects never had permanent 
project directors. In addition, the mid-academic-year startup date 
made creating new training classes impossible to arrange until the 
following fall semester, so up to 10 months elapsed before training was 
offered. To begin to determine the extent to which these problems 
persisted among FY1989 grantees, the survey asked whether the projects 
were able to start to provide services to clients immediately upon 
notification of awards and if not, how long thereafter. 

Most FY1989 grantees said their projects wou'^d not have existed 
without Federal support (see questionnaire, item 17). Of the 27 
projects, only four respondents indicated that the project would have 
existed without the Federal funds. Three of those were projects 
providing the services prior to the start of the grant. At the 
completion of the grant period, 14 projects expected to continue in 
their entirety, with another eight indicating that they would continue 
in a scaled-down form. Two projects reported that their Federal 
funding had not yet lapsed. 

Projects generally began to provide direct services to clients 
within a few months after their grants began, but a few did not provide 
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services for many months (see questionnaire, item 14). Excluding the 
three projects that were providing services to clients before the grant 
began, 12 projects began to provide services within three months of 
being awarded the grant, and nine more began within nine months. Three 
projects did not start, however, until 10 to 12 months after the grants 
were awarded (or six to eight months before the grants were originally 
due to end) . 

The grant period nonetheless appears to have been sufficient for 
many projects to have become institutionalized or to find other sources 
of support (see questionnaire, items 15 and 16). Twenty-two projects 
indicated they continued beyond the end of the grant. The most common 
sources of support among continuing projects were funds from the 
grantee institution (11 respondents), with private employers the second 
most common (eight respondents had commitments from employers and four 
were waiting for final agreements). Five respondents planned to charge 
students tuition for project services. 

The ability of projects to provide services soon after receiving 
an award appeared to have little or no effect on whether they continued 
services beyond the grant period (see Table F-22). In fact, there was 
a slightly greater tendency for projects that began providing services 
last to continue, with all of the latest-starting programs continuing 
in their entirety after the completion of the grant. Two of the three 
projects that indicated they were providing services before the grant 
started either received extensions to December 1991, or were no longer 
providing services at the time of the survey. 

Yet when asked directly about factors affecting their ability to 
complete their original plans, the short grant period was noted by the 
largest number of respondents (see questionnaire, items 59-61). Nine 
projects indicated they had encountered problems in completing their 
original plans. The most commonly cited reason for implementation 
problems was that the 18-month grant period was insufficient, a reason 
cited by five of these nine projects. Also noted by more than one 
project were difficulties in staff recruitment and/or retention (two 
projects) and planned activities that proved inappropriate (two 
projects) . 
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At the same time, however, 10 projects indicated that they had 
accomplished activities they had not originally planned (see 
questionnaire, items 62 and 63). The most commonly cited additional 
activity was curriculum development (five projects), followed by 
partner recruitment (four projects), and dissemination (three 
projects). Over half of these projects (six) indicated that they 
obtained additional funds that enabled them to undertake these 
unplanned activities. 

The Exportabil itv of Pro.iect Activities 

Because this was a demonstration program, respondents were asked 
what features of their projects had applicability for or could provide 
useful models for other educational or employment training programs. 
Respondents were provided with a list of possible project features and 
asked to rank up to three they considered applicable or useful for 
others. 

The projects were quite divided in choosing features of their 
programs that provided information for others (see questionnaire, item 
12). The most commonly selected first choice was customized training 
for a particular employer or group of employers, which was selected by 
six respondents. The second most commonly selected feature, selected 
by five respondents, was a new or improved curriculum. "New or 
improved kind of training," "established or strengthened public/private 
partnerships" and "applied high technology equipment to the delivery of 
training" each received four votes. Three projects chose "expanded 
access to training for disadvantaged or under represented groups." 
Only one selected "model of school -to-work transition," which was one 
of the few demonstration objectives specifically identified in the 
grants announcement. 

This wide range of responses can be read several ways. Given that 
this was intended to be a demonstration of public/private partnership 
for training in high-tech fields, the lack of consensus on exportable 
features is surprising. One might have expected the partnership or 
training choices to account for a large number of first choices, but 
they account for only eight of 27. The selection of customized 
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training as a first choice is surprising, because, by its nature, 
customized training is rarely applicable across industries or education 
institutions. 

If all three choices are added together, however, establishing or 
strengthening public/private partnerships does appear to be the feature 
most commonly selected as having widest applicability. Twenty projects 
considered partnership among the top three exportable features. Con- 
versely, few projects that did not select customized training as a 
first choice selected it second or third, making it one of the lowest 
ranked choices overall. Partnerships was the only exportable feature 
identified by more than half the projects, even with first through 
third choices are added together. 

The lack of consensus on what was being "demonstrated" is 
troubling. While it is not possible to reach conclusions from the 
responses to one question, it does seem fair to say that grantees did 
not share a common view of what they had used Federal support to "try 
out," And although all projects were, according to the grants 
announcement, supposed to "try out" public-private cooperation for 
high-tech training, only 20 projects considered their public/partner- 
ships, and only 13 projects considered their training as applicable to 
or providing a useful model for others. Nonetheless, it should be 
noted that 25 projects undertook some activities to disseminate the 
curricula they used. 
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Notes 

1. Two projects claimed that extensions to their projects made it impossible 
to provide needed data, but the evaluation staff indicated that final tallies 
of students or staff were not required. Some additional information was 
obtained from one nonresponding site at interviews conducted during a 
subsequent site visit. In order to maintain the integrity of the survey 
component of the analysis, however, the data summarized here are information 
collected from sites that returned project questionnaires prior to site visits 
and do not include information collected during the visits. 

2. Respondents were allowed to choose as many responses as were appropriate to 
their project. 

3. It is likely that these five were projects with more than one kind of 
training provided. 

4. The issue of industry and special populations is discussed in greater 
detail under the special populations heading. 

5. This final goal could be seen as an outcome of training. 

6. Some projects selected third and fourth ranked goals, but the lack of such 
goals across all projects make these responses difficult to analyze. 

7. This response probably means that the project provided computer-assisted 
basic skills instruction. 

8. We have not discussed these projects at length because additional inquiry 
about these projects yielded information that suggests high school students 
may not have been the primary clientele served. 

9. Of course, since many of these projects also report the fewest training 
hours per person, even if minorities were well represented they would not have 
received much training. 

10. Although described as basic skills in this item, the instruction is 
described as academic remediation elsev/here in the questionnaire. 

11. It is likely that this number does not include projects in which students 
were eligible for regular State or Federal financial aid, as an additional 
four projects indicated that financial aid was available through coordination. 
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